Wednesday, June 5, 2013

Facebook Removes Downloads of Your Posts

This post isn't exactly about math, but it is technical in nature, so I figured I'd get it out there. As part of my regular data-backup process, I routinely download my information archives from whatever online presences I can, such as Facebook (which I've been on since early 2010), Google Blogger (this blog you're reading right now), etc. Obviously on Facebook the thing that I'm most interested in is what I actually write, which are usually called "wall posts" (as opposed to photos or media, which I retain locally anyway). Once in a while I've found it very useful to pull up the downloaded posts file and search it for some particular bit of info, contact, or date. What I seem to have discovered is that sometime in the last few months, Facebook silently and completely removed our ability to download that "wall posts" information.

This first dawned on me the other day when I used the Facebook "Download Info" process (Gear icon > Account Settings > Download a copy of your Facebook data), and tried to search for a particular post. Well, the normal file was just entirely missing. You can see the difference below in the downloaded archive from March 2013 versus the download from June 2013. The file "wall.html" -- which actually contains all of my posts and is by far the largest data file in the old archive -- is missing from the new archive.




Now, initially I thought this was some kind of temporary glitch. (In the three years that I've been on Facebook, occasionally  the "wall.html" file has mistakenly contained just a few days worth of posts. Or for several months in 2012-2013 the download seemed to just fail completely any time I tried to use it.) But if I now go to the top-level "index.html" of the download, then I find that the reference-link to the wall posts has also been removed there ("Wall" used to be the second link but is now missing; before-and-after shots below):





So at this point I poked around a bit on Facebook's information pages, and landed on the page where they supposedly tell you what information is included in this and other resources ("Accessing Your Facebook Info", which I find at this link). This page currently describes three repositories of information: (1) "Downloaded Info", the archive which I've described above, (2) "Expanded Archive", another download which includes more transaction and login information, and (3) "Activity Log", which is an online-only manipulation of the Facebook timeline (not part of any download). What I see here is that "Your Posts" is now categorized under "Activity Log" (and note that this entry is also out of alphabetical order, possibly evidence of some change, and making it a bit harder to find in the list):



So what this means is that Your Posts, the things you've actually written on Facebook, are no longer included in the "Downloaded Info" which allegedly includes all of your info (and did from at least 2010-2013). The posts are not in the "Expanded Archive", either (I checked to be sure... it has dozens of files including Ad Clicks, Apps, Facial Recognition Data, Poke Data, Relationship Info, etc. ... but no wall posts). The "Activity Log" in which they are now categorized is online-only at Facebook, can't be downloaded, doesn't show all your posts at once, and can't be searched unless you know the date of the post that you're looking for in advance.  (I considered trying the Wayback Machine to find a date when this was altered on the help page, but of course Facebook bricks off any internet crawlers by way of its robots.txt file.)

In short: Facebook seems to have silently locked up everyone's personal posts in their system, with no way to get them out or search them, without any comment or notification of the switch that I can find anywhere online. The "Download Info" process screen itself remains unchanged, so potentially people could keep using it, not knowing that the largest and most fundamental type of data, their posts, has been stripped out of the archive.

Perhaps equally disturbing is how this hunt highlighted for me the fact that Facebook makes it totally impossible to search your own data (in any way other than tedious manual scrolling). I had flatly assumed that any digital entity would have this capability, if perhaps in a difficult-to-find location or UI. But Facebook apparently doesn't let you digitally search your own information in any way, and now they've removed the capacity to archive your information outside their system, too. Perhaps if attention is brought to this matter they might reverse course (as in some past cases) and restore the ability to truly "Download Info" from the largest and most fundamental aspect of your personal account.

Or can you now find any other way to download all the wall posts that you've written on Facebook?


Edit 6/13/13:  About a week now after I first posted this, and a fresh download does include the "wall" posts file. If this was a bug (see comments from Facebook associates, below), then we much appreciate this being resolved and hope the function sticks around in the future, too. Thanks!

29 comments:

  1. This may obey a perverse corollary of Hanlon's Razor, perhaps, "Do not attribute to malice what is adequately explained by lack of foresight." I wrote about a previous instance: http://blog.nexcerpt.com/2011/02/16/facebook-fail/

    In short, a few years back, Facebook suddenly stopped showing you all the posts of all your friends. Why? I doubt it was their stated goal of making your visit "more efficient." I strongly suspect they had overrun their capacity to calculate networks, and had to stop trying.

    Removing history would be a natural next step in trying to cut the glut they've created. I just requested my download, and am watching the message, "We're generating your personal archive. We'll email you when it's ready." That "Please wait" page is explicitly headed, "Get a copy of what you've shared on Facebook."

    It goes on to promise:
    "What's in your archive?
    Photos or videos you've shared on Facebook
    Your Wall posts, messages and chat conversations
    Your friends' names and some of their email addresses

    What's not in your archive?
    Your friends' photos and status updates
    Other people's personal info
    Comments you've made on other people's posts"

    "You can also download an expanded archive to see more about your account history." [That encoded URL says, "This expanded archive includes historic info like IP addresses we store about your logins to Facebook"]

    The "Learn what's included" link points here: https://www.facebook.com/help/405183566203254

    Regarding the lack of useful search, among Facebook's reasons for pushing "Graph Search," I suspect, is that it ~implies~ FB has a "search" function, without actually providing one.

    ReplyDelete
    Replies
    1. I hope you'll come back and report whether you got your posts ("wall.html") in your download! As it says above, it wasn't included for me, and the current "Accessing Your Facebook Info" page says it won't be there. (It certainly won't be in the Expanded Archive, but feel free to confirm that as well.)

      Delete
  2. You can search through your own posts via your activity log (link at the bottom right of your cover photo). I know this doesn't solve your current problem, but it might be useful!

    ReplyDelete
    Replies
    1. Yes, the Activity Log is pictured and discussed in the fifth paragraph above, and why it fails to be useful for this purpose.

      Delete
  3. If you go to your general account settings you should see this:

    Download a copy of your Facebook data.

    It is a link to a page that lets you download all your facebook data. Is this what you are missing?

    ReplyDelete
    Replies
    1. "Download a copy of your Facebook data" used to include wall posts. It no longer does. Try rereading the paragraph that begins "This first dawned on me the other day".

      Delete
    2. Yes, the second paragraph above likewise describes where the link is. The fact that they didn't change that page will trick people into not realizing that the download no longer contains your wall posts.

      Delete
  4. The ability to download my content was a key reason I kept using Facebook. To remove that without notice is particularly egregious.

    I'll be decreasing my Facebook use no with an eye to exiting.

    ReplyDelete
    Replies
    1. I totally agree with that. I thought this was extremely important.

      Delete
  5. You may be in luck, there are engineers at Facebook who want to fix it.
    https://www.facebook.com/antony.t.curtis/posts/10200541567835104

    ReplyDelete
    Replies
    1. Thanks for posting this. I hope someone bothers to reach out to Facebook for an official response.

      Delete
    2. Thanks so much for bringing this up to them, I appreciate it.

      Delete
  6. You can use ArchiveFB extension for Firefox, which saves your Wall, as far as I can tell, but not in an easy-to-navigate kind of way.

    If you can figure out how to get access to your friend's events, a feature which disappeared amid much complaint during the new timeline roll-out, that would be pretty helpful to me. Also kicking Zuckerberg in his shins would be pretty helpful to me.

    ReplyDelete
  7. I'm a Facebook engineer working on security and privacy. Thanks for pointing this out; it's definitely a bug. The fix should be live in a few days.

    The help page you found is intended to describe the most reliable location for accessing different types of data rather than an authoritative list of what's in your download archive. I'd love to see wall posts point at "downloaded info" on that help page, but as you point out the wall posts section hasn't been super reliable. The good news is that we're wrapping up some changes that should make this more reliable in the near future.

    ReplyDelete
    Replies
    1. Scott, thanks for bringing that up and posting here. I definitely appreciate the attention to the issue, and I'll look forward to seeing the fix when it's available.

      Delete
    2. And it does seem to be back in the download as of 6/13/13. Thanks again!

      Delete
  8. I've also noticed that your wall no longer contains everything that you've written on said wall. It's extremely frustrating when trying to look back at a status where you've asked friends a question ... you can't find their answers!

    ReplyDelete
  9. I tried testing this out, and the first thing I noticed was that wall posts are specifically mentioned in the text on the download page.
    Sadly, my archive is 650Mb and I have not managed to complete the download because of recurring failures explained only as a "network connection" error.
    Fortunately, I have been using an independent back up service since I joined FB.
    I also use a special tag to mark posts I know for sure I want to refer to. This tag is used as an identifier in my RSS feed. I use it in a simple IFTT.com recipe to send the status update (or photo etc) to my WordPress blog as a "private" post.

    ReplyDelete
  10. Looking forward to the day we are able to migrate one account to a new one with the data that would be really usefull.

    Downloading a copy now to see if the wall is back.

    ReplyDelete
  11. Ah - I finally got a Wall link and (as far as I can tell) my Wall data back to 2008, in my Archive Download from today, 6/22/13. I've been trying since mid-May, and couldn't find anything Wall. Good to see I wasn't alone. I thought I was going mad!

    ReplyDelete
    Replies
    1. There you go, same for me. It's always nice to have that "you're-not-insane" confirmation from somebody. :)

      Delete
  12. This might be related to your problem, but I just now noticed that archives include wall.htm, but all the links are stripped out. I'm checking to verify if this a bug or a regular feature, but that would really suck if it were true. (I should add that an archive from 3 or 4 months ago includes hyperlinks like normal, so this is a new development.

    ReplyDelete
    Replies
    1. Wow, that seems important, too. It just seems bewildering why these features are so flaky or variable or in transition or whatever.

      Delete
    2. I made a FB backup on 17 dec 2013, still missing all links and comments on wall.. dodgy.

      Delete
  13. I'm having the exact same problem.

    ReplyDelete
  14. Just downloaded my FB data and it's only giving me Wall posts back to the start of 2014.

    ReplyDelete
    Replies
    1. Interesting. The system does seem to be wildly variable. Maybe try in another month and see if it's all there (that's happened for me).

      But best if Facebook just fixed the damn thing. How their software can be so non-deterministic totally baffles me.

      Delete
  15. I've been trying to download my facebook data for months now. I go through the process, wait... and nothing. Ever. What gives?

    ReplyDelete
    Replies
    1. I've had cycles of that in the past. It did work for me when I requested it last month. Sorry to hear about that, very frustrating that it's so unpredictable.

      Delete