How Facebook is Adding an Identity Layer to the Internet

In what may become the next major privacy controversy for Facebook, the company has announced plans to automatically share certain information when a Facebook user visits certain “pre-approved” sites. In clarifying the feature, a spokesperson told VentureBeat that people should “think about Facebook Connect, but the user gets that experience when they arrive at the site rather than after clicking Connect.”

Given the way Facebook has repeatedly described “publicly available information” (PAI) since last fall’s privacy changes, this update is actually a logical next step for the company. Under a strict interpretation of Facebook’s policies, nothing would prevent a site from making use of such information already. Only technological barriers currently block the information flow – specifically, a site doesn’t automatically know who you are on Facebook when you visit.

At least, so it would seem. Researchers have already outlined ways that sites can infer a visitor’s social networking profile from other tracking mechanisms. In some ways, the new Facebook auto-connect simply builds on cookies and inline frames, the sources of earlier online privacy controversies. Furthermore, several security researchers have demonstrated exploits that led to data leakage. Nitesh Dhanjani demonstrated earlier this year that an authentication issue could give sites automatic access to the PAI of visitors, and just this week I reported to Facebook a vulnerability in their Platform that would allow sites to silently harvest all of a user’s profile information (details pending a patch).

Given the amount of data already flowing to Facebook applications and Facebook Connect sites (as well as their advertisers), the company’s moves towards more and more public sharing, and the history of privacy/security problems on the Facebook Platform, I’ve long argued that Facebook users should treat all of their content on the site as public. But Facebook has worked hard to maintain user trust, even making some content appear to be more private than it actually is. When I first discussed accessing public but hidden photo albums last December, I commented, “Making the albums hard to find gives an illusion of privacy and only delays any rude awakenings that may come from users who have inadvertently shared private photos.”

Now it may seem that Facebook users will finally understand the ramifications of default privacy settings. But the new system will probably be fairly subtle at first. Some users will find it creepy to be greeted on other sites by name, but such information will probably appear in a distinct, Facebook-labeled box (i.e., a Facebook Widget) to let a user know where the content comes from and make it still seem somewhat separate from the rest of the site. On the backend, though, the site will have access to the user’s public data.

What users may not realize is how much data they’re already sharing. This new style of Facebook Connect actually mirrors the behavior of Facebook itself. When you visit a Facebook application for the first time, it automatically knows who you are and can access your public data. (Correction: This only occurs in certain circumstances; more information here.) When you then click “Allow” to authorize the app, you give it access to all of your private data. Currently, an external web site knows nothing about you until you click “Connect.” If you do click, it has the same access to your private data as an authorized application. Now, Facebook is letting sites initially act like new applications by giving them access to your public data prior to full authorization.

In discussing the Facebook Platform, Anil Dash gave this analogy: “Think of the web, of the Internet itself, as water. Proprietary platforms based on the web are ice cubes. They can, for a time, suspend themselves above the web at large. But over time, they only ever melt into the water.” Depending on your perspective, either Facebook is finally melting into the water or the Web turned out to be the ice cube. With an automatic Connect system and the Open Graph API, Facebook is expanding its Platform to the rest of the Web. The only major difference between a Facebook-enabled web site and an actual Facebook application may soon be the URI.

You can start to get a sense of how this expansion may look by reading proposed changes to the service’s governing documents (see Inside Facebook’s excellent analysis):

We may also make information about the location of your computer or access device and your age available to applications and websites in order to help them implement appropriate security measures and control the distribution of age-appropriate content.

Currently, many sites hosting pornographic content will ask visitors to click a link verifying they are at least 18 or 21 before loading the material. With Facebook, the site could simply check your profile information first. Media companies worry about visitors accessing content outside of a given country; perhaps soon they can use your Facebook information to check your location.

Granted, providing fake details on your Facebook could easily foil some of these checks, but in many cases, that’s hardly different from lying about your age when you click or using a routing service to mask your location. Also, since if interact with friends on Facebook, you have a greater incentive to keep some information accurate. Facebook also reserves the right to terminate your account if you provide false profile information (despite also suggesting this strategy as a protection against identity theft).

My point is not to suggest that porn sites will soon be on Facebook’s “pre-approved” list or that Hulu would trust your profile over geographic IP data. I simply give these hypothetical scenarios to illustrate a larger trend: for better or for worse, your Facebook profile is becoming a virtual ID card.

Adding an identity layer to the Internet is not a new idea, but this may be the first time a system finds widespread adoption. Yet the Facebook identity model conflicts with many visions of how online identity should operate. “Open Stack” technologies, such as OpenID and OAuth, allow for federated setups. One of the first “Laws of Identity” by Kim Cameron states, “Digital identity systems must only reveal information identifying a user with the user’s consent.” Much of the consent in Facebook’s system comes from accepting the site’s terms at sign-up; many users will likely think that an opt-out Connect model violates Cameron’s principle.

And ultimately, user perception will be key to Facebook finding acceptance of its new endeavor. As social media researcher danah boyd discussed in her SXSW keynote, services with nothing technologically wrong can still disrupt social expectations (e.g. Google Buzz). (I rank the entire talk as must-read material for anyone working in the social networking space, but I’m only focusing on a few points here.) She also made a noteworthy distinction that I think will come up often as Facebook evolves:

Keep in mind that people don’t always make material publicly accessible because they want the world to see it….

Just because something is publicly accessible does not mean that people want it to be publicized. Making something that is public more public is a violation of privacy.

I think this distinction will be severely tested as the availability of Facebook data increases. I don’t dispute boyd’s evaluation, but coming from the perspective of security research, I know that when data becomes publicly available, it’s only a matter of time before it gets publicized in some way. With the wealth of information stored on Facebook’s servers, the site is becoming a favorite of both advertisers and attackers. Already we’ve seen hacks and tricks that make public Facebook data more public (see above), and each new site that integrates with Facebook is a new attack surface.

I’ve been cussed out by visitors to my site who think that by publishing weaknesses in the Facebook Platform or exposing seemingly hidden content I’m assisting those who maliciously hack people’s profiles. But much of what I post attempts to raise awareness of potential privacy and security issues before they get exploited by black hats. I can guarantee you I’m not the only one looking for Facebook weaknesses.

And that’s part of what concerns me about boyd’s distinction. The same technology that makes content “public” makes it easy to aggregate and publicize. For example, Pete Warden recently announced that he had built a dataset of 215 million Facebook profiles that he planned to publish for research purposes. Facebook eventually threatened to sue, prompting him to destroy the data, but no technology stands in the way of someone else recreating the dataset for their own purposes. In fact, with Facebook’s auto-connect system and the possibility of lighter rules for data storage, web sites may soon inadvertently recreate the dataset.

I honestly don’t think that Facebook is evil or that they care nothing about user privacy. Their new identity layer will likely bring benefits to many users and provide sites with valuable features. But just as Facebook became successful through providing users with a more private experience, the Internet became successful in large part because of its anonymity. While many users are happy with their personal Facebook account being a place “where everyone knows your name,” many users also value the rest of the Internet not knowing if they’re a dog. And as danah boyd put it so well, “No matter how many times a privileged straight white male technology executive pronounces the death of privacy, Privacy Is Not Dead.”

New Trick to View Hidden Facebook Photos and Tabs

Last December, I posted a bit of JavaScript known as a bookmarklet that allowed you to see photo albums for any Facebook user if the album privacy settings allowed it. This highlighted an example of “security through obscurity,” since the lack of links to photos on most profiles seemed to indicate no photos could be viewed. The trick worked as advertised, though it only displayed a few albums for those who had many.

The code came from my own experiments on accessing the hidden photos. It worked quite manually, retrieving data from a particular Facebook interface and stuffing it into the current page. I figured a more elegant solution could be found by re-using the code already embedded in the page, but I had not been able to sort out all of the built-in functions.

Last night and this morning, I found what I’d been missing before, and I now present a far simpler version that gives full access to all available albums of a given user. Simply bookmark this link (right-click and choose to add a bookmark) and click the bookmark when viewing someone’s profile on Facebook.

Once again, please note that this does not in any way circumvent a user’s privacy settings. If you mark your albums as visible only to your friends, this trick will not override that setting. I do not currently know of a way to access private photo albums, and if I did find one, I would report it to Facebook. My purpose in posting this code is to prove a point, not break into users’ accounts.

Here is the new source code:

javascript:(function(){CSS.removeClass(document.body, 'profile_two_columns');tab_controller.changePage("photos");})()

As I said, much simpler! I only had to find the right commands.

But the story doesn’t end there. This new method can be very easily adapted to load other information from a user’s profile, and the new possibilities raise more privacy ramifications. Once again, the trick does not actually override any settings, but it may break some user expectations and highlight the importance of overlooked or unknown settings.

The new behavior is that once can use similar code to access the canvas pages of applications the user has interacted with, as if the user had added the application as a tab on their profile. This includes the “Boxes” tab for users who have it. From what I understand, visibility of this tab page comes from the “Privacy” box under “Edit Settings” next to each application listed in a user’s Application Settings. Such controls have often been overlooked, particularly because they may not have seemed very relevant in the past. While many users stay aware of the privacy settings on their photos and wall posts, they may not think about the content they generate in the context of applications. Often, that content has little if any privacy controls applied.

Typically, any information available on an application tab is also available through the application itself, but this technique makes it far easier to find. However, it also raises some disturbing possibilities related to application data retention, and issue I’ve noted in the past but not seen discussed much elsewhere. For example, quite a while ago (as in months to years), I used the Pieces of Flair application with my personal Facebook account, arranging various buttons on my virtual corkboard. Eventually I pared down the number of applications I had authorized, and Pieces of Flair was one I uninstalled a number of months ago. Today, however, if you use the sort of bookmarklet posted above to check my Facebook profile for a Pieces of Flair tab page, you’ll see all my virtual buttons once again.

Facebook does notify applications when a user uninstalls them, but it’s up to the developer to actually do something about the data left behind. Apparently Pieces of Flair does nothing with the data, meaning a user has to manually delete their flair before removing the application if they want to truly get rid of the content they generated. Based on my experience, many applications behave in a similar fashion. Some may argue that this behavior is similar to Facebook “deactivating” an account, but at what point should the content expire, and how many applications offer a full deletion? Such issues become matters of retention policies, and based on my past studies of whether applications even had a privacy policy, I would guess that most applications do not currently have such terms.

All of this once again highlights the current complexity of data and privacy on the Facebook Platform. Granted, dealing with third-party applications is not a simple problem to solve, and I’m not simply criticizing Facebook for failing to build a perfect system. But these issues can very easily lead to unpleasant surprises for end users, and at some point someone will have to sort them out.

An Updated Guide to Backing Up or Exporting Your Facebook

Last update: March 15 at 7:11 p.m. EST

At the beginning of this year, I posted a series of FQL queries that would enable Facebook users to backup most of their account information. While a few services offering Facebook backups exist (see Part 6 below), I noted that none of them were anywhere near as comprehensive as using FQL when it came to messages and metadata. Over time, I added more queries and tricks to provide a more complete archive. Eventually, the only major block of information missing from the method was the e-mail addresses of friends.

I discovered that fellow blogger Tutkiun had found a way to exploit a Microsoft import tool to harvest friends’ addresses, but it was more obtrusive than I desired, since it required sending a Microsoft invitation message to all of those addresses during the import. At the time, though, the Tutkiun’s trick was the only legitimate way of exporting e-mail addresses.

In the last few weeks, though, Yahoo! has added an import tool to their e-mail product, and this now allows you to export your friends’ addresses quite easily and without sending any e-mails. Given this new feature and the slightly messy state of my last post, I decided to gather all of the current tricks for Facebook export and re-post them in this updated guide. Note that this guide can be rather technical, and most of the data provided will be in special formats that you likely won’t be able to browse and edit easily. If you’re looking for a simpler but less exhaustive approach, check out the services I discuss in Part 6 of this post.

I’ll note, though, that while this guide now gives you a very thorough export of their accounts, you may be disappointed to discover you have hardly anywhere to then import the information. Personally, I consider this method better suited for archiving your Facebook information than anything else. I think the best solution for “taking your social graph with you” will involve data portability which integrates with the Facebook API, but that will have to wait for another post.

I’ve divided my guide into seven parts. The first part can be accomplished using only the Facebook API Test Console. The second part involves a little trick for gathering phone numbers. The third section requires some technical know-how, as you need to administer a Facebook application. The fourth part describes using Yahoo! to get e-mail addresses. The fifth part provides option FQL queries for those wishing to archive even more of their information. The sixth section discusses solutions for backing up photos and videos, since the FQL queries only provide metadata and not the actual files. Finally, the seventh part adds some notes, tips, and limitations about the overall process. The second part yields data in JSON format, while the first, third, and fifth parts can either load JSON (my preference due to smaller file size) or XML. With the fourth section, Yahoo! can export to CSV format.

Update 1: Added optional queries 7-11, info on Fotobounce

Part 1: Standard FQL Queries

All of these queries can be executed using the Facebook API Test Console. Set the respone format to XML or JSON (I recommend JSON), select “fql.query” for the method, then fill in the query box. In each query, replace 00000000 with your Facebook ID number. (One way of finding this number is to visit your Facebook profile, right-click the link to “See All” of your friends, and copy the address. In the address, the number that follows “id=” is your Facebook ID.)

  1. Profile information of you and your friends: SELECT uid, first_name, last_name, name, pic_big, affiliations, religion, birthday, birthday_date, sex, hometown_location, political, current_location, activities, interests, music, tv, movies, books, quotes, about_me, hs_info, education_history, work_history, profile_url, profile_blurb, family, username, website FROM user WHERE uid = 00000000 OR uid IN (SELECT uid2 FROM friend WHERE uid1 = 00000000)
  2. Your friend lists: SELECT flid, name FROM friendlist WHERE owner = 00000000
  3. The members of your friend lists: SELECT flid, uid FROM friendlist WHERE flid IN (SELECT flid FROM friendlist WHERE owner = 00000000)
  4. Pages you’re a fan of: SELECT page_id, name, pic_big, website, type FROM page WHERE page_id IN (SELECT page_id FROM page_fan WHERE uid = 00000000)
  5. Links you have posted: SELECT link_id, owner_comment, created_time, title, summary, url, image_urls FROM link WHERE owner = 00000000
  6. Events you have attended: SELECT eid, name, tagline, pic_big, host, description, event_type, event_subtype, start_time, end_time, creator, location, venue FROM event WHERE eid IN (SELECT eid FROM event_member WHERE uid = 00000000 AND rsvp_status = “attending”)
  7. Your notes: SELECT note_id, title, created_time, content FROM note WHERE uid = 00000000
  8. Comments on your notes: SELECT object_id, post_id, fromid, time, text FROM comment WHERE object_id IN (SELECT note_id FROM note WHERE uid = 00000000)
  9. Your photo albums: SELECT aid, cover_pid, name, created, modified, description, location, size, link, visible, modified_major, type, object_id FROM album WHERE owner = 00000000
  10. Comments on your photo albums: SELECT object_id, post_id, fromid, time, text FROM comment WHERE object_id IN (SELECT object_id FROM album WHERE owner = 00000000)
  11. Your photos’ metadata: SELECT pid, aid, src_big, src_big_height, src_big_width, link, caption, created, modified, object_id FROM photo WHERE aid IN (SELECT aid FROM album WHERE owner = 00000000)
  12. Comments on your photos: SELECT object_id, post_id, fromid, time, text FROM comment WHERE object_id IN (SELECT object_id FROM photo WHERE aid IN (SELECT aid FROM album WHERE owner = 00000000))
  13. People tagged in your photos: SELECT pid, subject, text, xcoord, ycoord, created FROM photo_tag WHERE pid IN (SELECT pid FROM photo WHERE aid IN (SELECT aid FROM album WHERE owner = 00000000))
  14. Your videos’ metadata: SELECT vid, title, description, thumbnail_link, embed_html, updated_time, created_time FROM video WHERE owner = 00000000
  15. Comments on your videos: SELECT object_id, post_id, fromid, time, text FROM comment WHERE object_id IN (SELECT vid FROM video WHERE owner = 00000000)
  16. People tagged in your videos: SELECT vid, subject FROM video_tag WHERE vid IN (SELECT vid FROM video WHERE owner = 00000000)
  17. Groups you’re a member of: SELECT gid, name, nid, pic_big, description, group_type, group_subtype, recent_news, creator, update_time, office, website, venue, privacy FROM group WHERE gid IN (SELECT gid FROM group_member WHERE uid = 00000000)

Part 2: Phone Numbers

Visit this URI: http://www.facebook.com/friends/ajax/superfriends_phonebook.php?__a=1 The “payload” parameter lists any phone numbers of Facebook friends you can access, sorted by the Facebook ID number of each friend.

Part 3: Extended FQL Queries

These queries can only be executed by an application with certain extended permissions, and the Test Console does not have any. To perform these queries on your own, you need to have a Facebook application you administer. You can then enable the needed permissions using the following two URIs, replacing “ffffffffffffffffffffffffffffffff” with the application’s API key:

  • http://www.facebook.com/authorize.php?api_key=ffffffffffffffffffffffffffffffff&v=1.0&ext_perm=read_mailbox
  • http://www.facebook.com/authorize.php?api_key=ffffffffffffffffffffffffffffffff&v=1.0&ext_perm=read_stream

Once these permissions are enabled, you can again use the API Test Console by setting the application field accordingly.

  1. Threads in your inbox (requires “read_mailbox” permissions): SELECT thread_id, folder_id, subject, recipients, updated_time, parent_message_id, parent_thread_id, message_count, snippet, snippet_author, object_id FROM thread WHERE folder_id = 0
  2. Threads in your outbox (requires “read_mailbox” permissions): SELECT thread_id, folder_id, subject, recipients, updated_time, parent_message_id, parent_thread_id, message_count, snippet, snippet_author, object_id FROM thread WHERE folder_id = 1
  3. Messages in your inbox (requires “read_mailbox” permissions): SELECT message_id, thread_id, author_id, body, created_time, attachment FROM message WHERE thread_id IN (SELECT thread_id FROM thread WHERE folder_id = 0)
  4. Messages in your outbox (requires “read_mailbox” permissions): SELECT message_id, thread_id, author_id, body, created_time, attachment FROM message WHERE thread_id IN (SELECT thread_id FROM thread WHERE folder_id = 0)
  5. Your wall posts (requires “read_stream” permissions): SELECT post_id, app_id, source_id, updated_time, created_time, attribution, actor_id, target_id, message, app_data, action_links, attachment, comments, likes, privacy, permalink, tagged_ids, is_hidden FROM stream WHERE source_id = 00000000
  6. Comments on your wall posts (requires “read_stream” permissions): SELECT post_id, fromid, time, text FROM comment WHERE post_id IN (SELECT post_id FROM stream WHERE source_id = 00000000)

Part 4: E-mail Addresses

This part requires you to have a Yahoo! Mail account. If you don’t already have one, you can create one for free. In fact, I’d advise creating a new account to avoid your Facebook friends’ e-mail addresses getting mixed up with any others already in your address book.

  1. To add your friends’ e-mail addresses to your Yahoo! Address Book, follow the steps given on this page at the Yahoo! Mail blog. Essentially, you simply click “Import Contacts,” choose “Facebook,” and follow the steps. You will have to authorize a Facebook application built by Yahoo! for this purpose.
  2. To save a local copy of these addresses, you can use the export tools in Yahoo! Address Book. This help page from Yahoo! provides information on ways to transfer contacts from Yahoo! to various other programs. These specific steps are given for saving your entire address book as a CSV file:
  1. Open your Yahoo! Address Book and click “Import/Export” in the upper-right corner.
  2. In the Export section, click “Export Now” next to the phrase “Microsoft Outlook.”
  3. A dialog window opens and gives you the option to save your Yahoo! Address Book to disk as a .csv file. Click “Save to Disk”.
  4. Click “OK”.

Part 5: Optional FQL Queries

These FQL queries can be in the same way as those from Part 1, since they do not require any extended permissions. Personally, I think they go somewhat beyond the scope of what I’d consider an account backup, but people have shown an interest in exporting more information and none of these queries provide content not already accessible to you via the Facebook site. By the way, I’m not aware of a way to access who was tagged in notes and thus save the notes you’re tagged in.

  1. Photos you’re tagged in: SELECT pid, aid, src_big, src_big_height, src_big_width, link, caption, created, modified, object_id FROM photo WHERE pid IN (SELECT pid FROM photo_tag WHERE subject = 00000000)
  2. Comments on photos you’re tagged in: SELECT object_id, post_id, fromid, time, text FROM comment WHERE object_id IN (SELECT object_id FROM photo WHERE pid IN (SELECT pid FROM photo_tag WHERE subject = 00000000))
  3. People tagged in photos you’re tagged in: SELECT pid, subject, text, xcoord, ycoord, created FROM photo_tag WHERE pid IN (SELECT pid FROM photo WHERE pid IN (SELECT pid FROM photo_tag WHERE subject = 00000000))
  4. Videos you’re tagged in: SELECT vid, title, description, thumbnail_link, embed_html, updated_time, created_time FROM video WHERE vid IN (SELECT vid FROM video_tag WHERE subject = 00000000)
  5. Comments on videos you’re tagged in: SELECT object_id, post_id, fromid, time, text FROM comment WHERE object_id IN (SELECT vid FROM video WHERE vid IN (SELECT vid FROM video_tag WHERE subject = 00000000))
  6. People tagged in your videos you’re tagged in: SELECT vid, subject FROM video_tag WHERE vid IN (SELECT vid FROM video_tag WHERE subject = 00000000)
  7. People who liked your notes: SELECT object_id, user_id FROM like WHERE object_id IN (SELECT note_id FROM note WHERE uid = 00000000)
  8. People who liked your photos: SELECT object_id, user_id FROM like WHERE object_id IN (SELECT object_id FROM photo WHERE aid IN (SELECT aid FROM album WHERE owner = 00000000))
  9. People who liked your videos: SELECT object_id, user_id FROM like WHERE object_id IN(SELECT vid FROM video WHERE owner = 00000000)
  10. People who liked photos you’re tagged in: SELECT object_id, user_id FROM like WHERE object_id IN (SELECT object_id FROM photo WHERE pid IN (SELECT pid FROM photo_tag WHERE subject = 00000000))
  11. People who liked videos you’re tagged in: SELECT object_id, user_id FROM like WHERE object_id IN (SELECT vid FROM video WHERE vid IN (SELECT vid FROM video_tag WHERE subject = 00000000))

Part 6: Photos and Videos

The FQL queries in previous sections relating to photos and videos save metadata (e.g. caption, size, date uploaded, etc.) but not the actual files themselves. For photos, one field (src_big) provides a direct URI for downloading the file, and programmers so inclined could build scripts that process the JSON data and download all of the referenced files.

Several other options already exist, though, for downloading photos. Currently I know of these:

  1. Backupify. This service provides online backups of many online sites besides just Facebook. Their basic plan is free and provides backups for one account per site (e.g. one Facebook account, one Twitter account, etc.). If you find my backup method too technical, Backupify’s Facebook archives include an XML file that contains a list of your friends (with birthdays and locations), your notes, your status updates, your links, and events you’ve attended. Backupify also saves a copy of all your photos and all photos you’re tagged in. With the free plan, your photos are stored online and you can download them individually, but you have to pay for a plan that offers ZIP archives for downloading multiple files at once (currently $39.95/year, though free on sale through March 21).
  2. SocialSafe. This is an application you install on your computer which uses the Facebook API to create local backups. If you find my backup method too technical, SocialSafe offers the advantage of an interface for easily browsing your backups and maintaining different versions over time, though at a small fee ($2.99 at time of publication). Currently, SocialSafe only backs up your profile information, a list of your friends, your wall posts, and your photos, but the developers have stated they plan on adding more data, and upgrades are free after your purchase. Note that SocialSafe only saves photos you have uploaded; their site says they removed the ability to download tagged photos for legal reasons. I don’t know the details of the situation, but it seems they wanted to avoid liability in case you are tagged in copyrighted photos.
  3. FaceDown. This is a free program you can download which mimics a web browser logging into Facebook and lets you save all of the photos linked from a given Facebook page. I’ll quickly add that this program violates a fundamental security practice in that it asks for your Facebook username and password directly. Normally I would say you should never provide such credentials to a non-Facebook service or program, but in this particular case, your options are somewhat limited and the program functions as a web browser (i.e. it does not store your credentials or forward them to a third party). Also, I’m not entirely sure if this program remains consistent with the Facebook terms of service, which forbids “automated means” of collecting information from other users. I know Facebook monitors such activity, so while it’s unlikely if you’re careful, using this program could risk your account being disabled by Facebook.
  4. ArchiveFacebook. This is a Firefox add-on (it requires the Mozilla Firefox web browser) which seeks to archive a variety of information from your Facebook account, including photos. However, I’m quite certain this add-on violates the Facebook terms of service, which forbids “automated means” of collecting information from users, and thus I cannot recommend it. Using this add-on will risk your account being disabled.
  5. Fotobounce. This is an application you install on your computer (Mac or Windows) that syncs local photo albums with those on Facebook, and even maintains tags in photos. It appears that if your Facebook were somehow deleted, this program would be the easiest for restoring photo albums. From what I understand, Fotobounce also offers export capabilities similar to other programs listed above.

Obviously, no option is perfect, but of the four, I would personally say that Backupify is the one I find most appealing – except that I’m intrigued by Fotobounce, which I only recently discovered. I should note that I have personally used only Backupify and SocialSafe.

For videos, I know of no simple way to download multiple files. However, I have created a JavaScript bookmarklet which loads the source MP4 file of a given Facebook video. To use it, bookmark this link in your web browser and load the bookmark when viewing a page for a Facebook video. The bookmarklet automatically loads a high quality version if it’s available. The source code for the bookmarklet is as follows:

javascript:(function(){x=document.getElementsByTagName('div');for(y=0;y<x.length;y++){if(x[y].className.indexOf('mvp_player')==0){z='swf_'+x[y].id;}}eval('x='+z+'.getVariable("highqual_src");');if(!x){eval('x='+z+'.getVariable("video_src");');}x=x.replace(/[%]3A/,':');x=x.replace(/[%]2F/g,'/');location=x;})()

Alternatively, I discovered that others have built a Firefox add-on (requires the Mozilla Firefox web browser) called Facebook Video which lets you download Facebook videos. I have not tested this add-on or investigated it much, but it certainly appears to be legitimate.

Part 7: Additional Notes

  • With each FQL query, I have selected what I thought would be the most useful fields and what would yield reasonable response sizes. While I tried to make each request complete enough to represent a backup of the relevant information, you may disagree with my choices.
  • Some of the FQL queries may generate very large responses, so wait a few minutes if the results do not appear right away. In some cases, you may need to limit requests (i.e. adding “LIMIT 1,500″ for the first 500 results, “LIMIT 501,1000″ for the second 500, etc.). I tested the first query in Part 1 against an account with about 700 friends and did eventually receive a response using the Test Console, but it was about 1.2MB of data.
  • In my testing, the responses to these queries were complete, with one apparent exception. The query for comments on wall posts seemed to include only recent comments. I haven’t investigated the issue much yet but will post any updates if I find more information on the issue.
  • Note that the optional queries in Part 5 will generate data that overlaps significantly with information from Part 1. For example, any photo that you are tagged in which you also uploaded will appear in queries from both sections.
  • I have tested all of these techniques and they worked for me as described, but I can make no guarantee that they will work for you or continue to work in the future. I cannot guarantee your use of these methods will not result in any loss of information or exposure of personal information. Also, while I believe all of these methods follow any applicable terms of service at the time of publication (unless otherwise noted), I cannot guarantee that they do nor that they will be permitted in the future. You should not use these methods to download any content that you are not licensed to download. All of these instructions are provided simply as a convenience, and you follow them at your own risk.
  • Please feel free to send questions or feedback to me (my e-mail is theharmonyguy@gmail.com), but realize I may not be able to provide technical support if you’ve never used FQL or worked with JSON data. This is not a general solution for non-technical users by any stretch, but it at least provides an option for backing up Facebook information that’s more complete than methods I’ve seen elsewhere.

Facebook Adds Code for Clickjacking Prevention

Over the last several months, many Facebook users have fallen prey to clickjacking “worms.” Lured by tempting links on a friend’s wall, victims would click through to a page that seemed to promise interesting photos or other info. But the page instead contained an invisible inline frame that loaded Facebook’s share page. When a user clicked for their prize, they instead posted the attack page to their wall as well. In at least one case, the attack page also tried to install malware.

In each case, Facebook responded fairly quickly, and one benefit to the site’s centralized nature is that administrators can purge known links to clickjacking attacks from walls across the system. Still, by the time such problems become known, myriad users may have already been compromised. Posting shared links may not cause much damage, but this blog has outlined before how much is truly possible with clickjacking. It’s long been possible for attackers to use clickjacking for installing applications, thus harvesting user data before Facebook cuts off viral channels. As with many security risks, though, there seems to be a lag time between discovery of potential and actual exploitation. Even the basic clickjacking attacks of late have been possible for quite a long time before they first surfaced.

Given the many threats posed by clickjacking, I’ve been surprised that Facebook has never seemed to show an interest in implementing code aimed at blocking an attack. After similar “worms” appeared on Twitter, the microblogging site added framebusting JavaScript to reduce the risk. Completely avoiding clickjacking is difficult (if not impossible apart from added browser protections), but such measures certainly make it much more difficult.

But quietly, Facebook has fortified their code. I’m not sure how long the new protection has been in place – I’ve not seen it reported anywhere, and only noticed it this week. The only mention I saw of it on Twitter came only a few days previous. In any event, Facebook deserves praise for the change, and I personally find their current solution rather clever.

On high-risk pages (possibly every page, but I’ve only checked high-risk ones, such as for link sharing and application authorization), a block of code checks whether the page is “top” – that is, whether or not it’s inside of a frame. If the page finds itself “framed,” an image is loaded that notifies Facebook, and a div element is loaded on top of the page. The div is set to cover every element in the page, and adds a dark filter if visible. Finally, the div has an onclick event set which loads the Facebook page outside of the frame. Thus if someone clicked a link hiding an invisible Facebook iframe, they would only click the div and see the page reloaded in the full window.

An immediate weakness is that this requires JavaScript to work, but you’ll find it’s rather hard to use Facebook without JavaScript enabled. That may be disappointing from a usability perspective, but it’s certainly a plus in this context. In particular, authorizing an application appears to require JavaScript.

I’m very glad to see Facebook add an innovative way of protecting their users from clickjacking attacks. This change adds a layer of difficulty to several Facebook attacks I’ve described in the past. Granted, there are still many ways that applications can be exploited, but this new code may remove at least one attack vector.

Facebook StumbleUpon Digg Twitter Instapaper FriendFeed Delicious Google Bookmarks Share/Bookmark

Introducing Social Hacking’s New Look—and Myself

I’m happy to make several announcements today. First, I’ve long felt this blog had a rather staid design that needed upgrading. Over the last several weeks, I’ve worked on putting together the new look you now see at theharmonyguy.com. I went ahead and brought the theme live, but I still plan on making further adjustments to the code, so I’d ask for patience as the site developers. Thanks to Elegant Themes for providing the basis of the new design. I have some ideas for further updates to the content of this site to match the theme change, but those will have to wait until later.

Second, I’d like to introduce myself. I’m known to many online as “theharmonyguy,” a screen name that goes back many years for me. Using it as my moniker for writing about security research was a split-second decision when TechCrunch covered my first major “hack” in 2007. Part of my decision came from wanting to keep my hacking endeavors separate from other development projects I had in mind back then. More recently, though, security research has become more than a small hobby, and I think it’s time to shed the anonymity. While I’ll continue to use “theharmonyguy” as an online identity, my real name is Joey Tyson. I graduated from Wake Forest University last year with a masters degree in mathematics, but I’ve spent several years working in IT consulting and web development prior to my career as a hacker.

And that brings me to my third announcement. I’ve officially joined the team at Gemini Security Solutions in Chantilly, Virginia, and look forward to starting work with them in March. A big shout-out to the Liquidmatrix Security Digest for the job posting that led me to Gemini. I’m excited about serving Gemini as they provide quality information security consulting to other companies. Also, I’ve been graciously allowed to continue this blog and my personal Twitter feed with the caveat that they don’t interfere with my work duties. Please note, however, that everything I post here is my own perspective and does not in any way reflect on my employer.

Over the next few weeks I’ll be moving to a new state, adjusting to a new area, and getting settled in a new job, so I may not be posting as frequently during the transition. But I still plan on maintaining (and perhaps expanding) both this blog and my Twitter feed for the near future. Thank you so much to all my readers for your help and support!

Facebook StumbleUpon Digg Twitter Instapaper FriendFeed Delicious Google Bookmarks Share/Bookmark

Using Google Buzz Can Expose Your Gmail Address

I’ve discovered another trick that may surprise some, this time relating to Google’s services. I don’t view the issue as a vulnerability, but it likely goes against user privacy expectations. In short, having a public Google profile (which you might have created when checking out Google Buzz) can allow others to figure out your Gmail address.

This really shouldn’t be that surprising, given that your username is generally consistent across Google services, and a public profile is public. But those who currently have numeric profile addresses (e.g. http://www.google.com/profiles/104424237445852766735) might think their profile is not easily tied to their username.

But by using Picasa, Google’s photo sharing service, it’s often quite simple to go from a numeric profile address to an actual username. To protect yourself from this access, visit the Picasa settings page.  Under “Your gallery URL,” add a new username and select the new username for your gallery URL. Also, you may want to edit your nickname.

In my testing thus far, it matters little whether you’ve used Picasa before – if you have a Gmail account, Picasa is also enabled on your account. And while individual Picasa albums have privacy controls, I have not found a way to block simply loading your Picasa home page.

With the introduction of Buzz, Google is encouraging users to take advantage of Google profiles. But in the process, Google is tying together services that many users may have treated quite distinctly in the past. If you want your Gmail address to remain private, you need to manage properly the other Google services you use to avoid one of them exposing your Gmail username.

Update (Feb. 13): It appears Google has adjusted their services to prevent the original URI trick from working. Previously, adding a profile number to picasaweb.google.com (e.g.  http://picasaweb.google.com/104424237445852766735) would either load a page with the username visible, the username embedded in the page’s source code (_user.name in JavaScript), or an error page in a few particular instances. One configuration that would simply produce an error page was if you had Picasa setup under a different username than your Gmail username, hence my advice. It now seems that using a numeric Picasa URI will either load an error page if the user does have Picasa setup or a page indicating the user does not have Picasa galleries but with no username anywhere in the page.

I’ve already done some preliminary testing to see if Google Reader could also be used to discover usernames, but so far that does not seem possible. Still, it’s wise to be cautious when using a tool that interacts with so many other services.

Facebook StumbleUpon Digg Twitter Instapaper FriendFeed Delicious Google Bookmarks Share/Bookmark

Facebook’s Fluid Definition of Publicly Available Information

In yet another example of security through obscurity, Facebook modified their platform last July to prevent applications from accessing public photo albums for users that were not friends of the logged-in user. Facebook had previously said such applications did not violate the site’s privacy policy, since the behavior followed photo album privacy settings – applications could only load albums marked as visible to “Everyone.”

But “Everyone” is the default privacy setting for photo albums, and many users probably don’t mean for everyone to see their photos. As a CNET report noted:

A Facebook spokesperson said the company made the change so the technology more closely matched users’ privacy expectations.

“We made this change in order to ensure that users who have their profiles set to a privacy other than ‘everyone’ are not surprised by photos being exposed through the API,” Facebook engineer Matt Trainer wrote in response to complaints on the developer forum site.

In other words, Facebook introduced inconsistent application of privacy settings (are the albums available to everyone or not?) so that users would continue to believe a false representation of who could access their content.

Fast forward to 2010, as Facebook users grapple with revamped privacy controls, new default settings, and the general introduction of “publicly available information,” or PAI. With the announcement of PAI, Facebook removed users’ ability to control access for certain bits of information. Among the data now included in the PAI category: the list of your Facebook friends.

That particular change riled many critics, and Facebook eventually backpedaled a bit, allowing users to remove friends lists from their profiles. But the company made quite clear that your list of friends was still considered publicly available information. With this behavior, Facebook setup a strange distinction between permission and visibility. Everyone was technically allowed to see your friends list, but had no means to do so if you removed it from your profile.

Of course, it wasn’t long before someone discovered a “means to do so.” In December, I posted a simple trick that would reveal the names and profile pages of any user’s friends, regardless of whether they blocked such a list on their profile. I try to follow principles of responsible disclosure with security vulnerabilities, but in this case, my “hack” in no way violated or worked around Facebook’s stated privacy policy, since friends lists were now public.

But the other day, I tried using my trick once more, and noticed that it no longer worked for users who chose to hide their friends lists. I’ve also found that issuing an FQL query for the friends list of a user beside the currently logged-in user fails – I don’t recall precisely the behavior of such a command back in December.

Oddly enough, Facebook has yet to block my trick for viewing a user’s public photo albums, which avoids last July’s changes as it does not involve the Facebook API.

It seems Facebook wants to have their cake and eat it too – give users the impression they still maintain control over their data, but still classify the data as public if circumstances warrant. Personally, I think it better for the company to treat “public” information consistently so that any user surprises come now and not later when people discover other means of accessing content.

By the way, a simple adaptation of my photos trick lets you discover a user’s full name based on their profile ID (which, by the way, is included in the filename of every photo you post – and that filename may be maintained if you upload the photo to sites such as Twitter), regardless of their profile privacy. (Some users restrict access to their profile, so trying to load it directly or request their name via the Facebook API Test Console would fail.) Is this new trick a violation of user privacy or a demonstration of “publicly available information?”

Facebook StumbleUpon Digg Twitter Instapaper FriendFeed Delicious Google Bookmarks Share/Bookmark

Cross-Site Scripting Pop Quiz

You have ten seconds to spot the problem in the image below. Ready? Go!

Example of ESPN's "Report a Bug" page

I hope you spotted the problem right away, as it’s a classic example of a cross-site scripting hole. The page mentions that the report will reference a particular URI, and that address also appears as a parameter in the page’s URI. As you might guess, the parameter is not being filtered, allowing one to insert any HTML code.

I found it rather ironic that I came across this problem as I was looking for a means to contact ESPN about two other XSS holes. All three issues were reported to ESPN back in late November, then reported again via different means earlier this month. After receiving no response to either report, I decided to go ahead and release this hole publicly.

By the way, I realize some of my posts about XSS issues aren’t directly related to social networking sites and thus diverge from the usual fare on this blog. However, I think they can serve as important lessons for all developers, including those building social networking applications. This sort of vulnerability is exactly the type that leads to FAXX hacks in Facebook applications. And perhaps it will serve as some comfort to smaller developers that even large sites are susceptible to such problems. Anyway, I also think it’s important to record these finds for future reference, and this blog is about the only place I have to do so.

Facebook StumbleUpon Digg Twitter Instapaper FriendFeed Delicious Google Bookmarks Share/Bookmark

1 2 3 4 5 6 15