Facebook’s indefensible, unscholarly research [Updated]

Note: This blog post has been updated 5 times since I first published it less than 24 hours ago. The updates are inserted where they make the most sense, so read through the post to get them all. They are bolded to make it easier.  

Note #2: I’ve summarized the latest information in a new post


I’m supposed to be on hiatus. And I was, until I woke up this morning and clicked a link to an A.V. Club article describing a Facebook-based research paper on contagious emotions. I read the article and was dumbfounded. Then I thought, well, I should make sure it’s being reported accurately, so I downloaded the paper and read it. That didn’t help. Indeed, it made things worse.

Facebook has billions of users and is now a publicly traded company whose financial health depends on keeping shareholders happy. One of the ways it does this is by constantly tweaking its presentation of personal, public, and commercial information in order to increase participation and maximize ad revenues. Facebook has never been demonstrably interested in its users’ privacy, and unlike Google, it has never had “don’t be evil” as a maxim. Its founder is on record as saying that wanting to have more than one online/offline identity displays a lack of integrity. For Facebook, information is money, i.e., your information mints their money.

I dumped my Facebook account years ago after barely using it, because I got tired of having to be vigilant about my privacy settings every time they tweaked the site (and they tweak the site a lot). When I teach a privacy course, Facebook is one of the obvious examples of an online world that makes privacy control extremely difficult. I tell my students and anyone else who will listen, if you’re going to use Facebook, set your privacy options to maximum and hope for the best.

But the story that broke today is not about privacy. It’s about integrity, trust, and informed consent. Briefly stated, one of Facebook’s “data scientists” coauthored a paper with two academics that reported the results of a large-scale experiment in which people’s feeds were manipulated in order to test whether adding and subtracting positive and negative messages affected readers’ emotional responses at the time they read the post and in the days following. The A.V. Club story has the guts of the research and links to the paper. In addition, The Atlantic has two articles, one an interview with the psychologist who edited the article for the journal and another summarizing some of the points. In addition, Slate has an excellent article on the ethics of the research, with contributions from ethicists and lawyers.

As the Slate and Atlantic articles suggest, there is no way that what the researchers claim to have obtained qualifies as “informed consent” under either the Common Rule or most universities’ IRB (Institutional Review Board) procedures. I find it interesting that in the author credits to the article, the Facebook researcher is credited with the data collection and data analysis, while the two academic authors (one from UCSF and the other from Cornell University) are responsible only for the research design and writing up the paper. This seems to silo them from IRB requirements (which understandably focus on the treatment of human subjects during data collection and analysis). Here is Cornell University’s FAQ on IRB procedures. Under their procedures, there is no way these data could be collected and analyzed without IRB scrutiny, and it has to take place before the study can begin.

[Update: Lee (@ZLeeily) contacted Fiske directly and has posted the results of her email exchange. It is unclear which of the two universities’ IRBs approved this, since Fiske doesn’t name the university. [ETA: but see Update #4 below.] The board seems to have treated this manipulation as functionally equivalent to editorial & advertising-oriented curation of users’ feeds, which is dumbfounding to me. It certainly doesn’t seem to fit the points in Cornell’s FAQ, but individual decisions can obviously diverge from these.]

[Update #2: The editor of this article also edited the controversial and highly criticized article on “female” hurricanes.]

In the article, the authors assert that the data collection and analysis technique

“was adapted to run on the Hadoop Map/Reduce system (11) and in the NewsFeed filtering system, such that no text was seen by the researchers. As such, it was consistent with Facebook’s Data Use Policy, to which all users agree prior to creating an account on Facebook, constituting informed consent for this research.”

This is hogwash, for two reasons.

  1. Whether data are collected by humans or by automated procedures has no bearing on informed consent conditions. This may matter to Facebook in terms of what it promises its users, but it is irrelevant to academic human-subject rules. Anonymization is relevant and was followed here, but that’s a separate issue.
  2. Facebook’s own Data Use Policy stipulates that “in addition to helping people see and find things that you do and share, we may use the information we receive about you … for internal operations, including troubleshooting, data analysis, testing, research and service improvement.”

Note that it says “information we receive about you.” Also note that this information will be used “for internal operations.”

A publicly accessible journal article is a long way from “internal operations.” And this data was not passively received, it was experimentally manipulated. Some people received positive messages, some negative. The entire experiment was designed to test whether users’ emotional states appeared to be altered as a result of this manipulation. (I say “appeared” because the data are collected from users’ Facebook posts; there are no direct measures of individual emotional states).

Look, academics experimentally manipulate emotions in studies on a regular basis. Academics are also doing Facebook-based studies of politics and other topics. I’ve personally conducted dozens of interviews for two different projects, both of which ask difficult personal questions of my subjects, and those interviews can have emotional implications as well. The problem here isn’t what the study wanted to measure. It’s that they didn’t ask people whether they wanted to participate.

Even in “deceptive research,” i.e., research where the participants cannot be told of the goals of the research in advance because it may taint their behavior and therefore the results, there are protocols. These include debriefing the subjects as soon as possible after the manipulation and allowing them to withdraw their data if they so desire. None of these protocols appear to have been followed.

Facebook is not required to conform to the Common Rule because it is unlikely to have received government funds, given that it has money to burn on studies like this.

[Update #4: Law professor James Grimmelmann notes (via @jonpenney) that the study seems to have had at least some federal support: a Cornell Chronicle story reports that it was funded in part by the James S. McDonnell Foundation and the Army Research Office. The latter is a federal institution and therefore covered by the Common Rule. One of the academic co-authors is currently at Cornell and the other, while now on the UCSF faculty, was a postdoc at Cornell when the study began. That makes it more likely that Cornell’s IRB approved the research. ]

[Update #5: According to Forbes reporter Kashmir Hill, who has been doing stellar reporting on this story, the IRB review was not at a university but within Facebook itself. So we’re back to where we started. After 5 updates, I think it’s time for a new post. This is getting more confusing, not less. ]

But unlike some writers, I’m not completely convinced this study qualifies as legal in every subject’s country. The TOS are vague and broad, and in this case not only is there no reason for Facebook users to think “research” includes “emotional manipulation,” Facebook seems to have violated its own TOS assurances by reporting the results to the world. And again, these data weren’t just “received,” they were created by the experiment. In the absence of the experiment, the statistically significant results would not have obtained. Given that users were selected on the basis of using English as their language, the laws of multiple countries are relevant, including countries with varied legal definitions of privacy and consent.

[Update #3: Also from @ZLeeily: we don’t know whether the researchers excluded minors from their sample. IRB rules often require that researchers treat minor and adult human subjects separately.]  

But even if this research was legal (and that’s a big if), it wasn’t ethical. I don’t know anyone who loves IRB reviews, but I know very few scholars who seriously want to go back to the bad old days of every researcher abiding by his own code. IRBs frequently overreach and are tone deaf, but so are researchers.

The history of human subject research is full of horrible examples of abuse and exploitation. As an empirical social scientist, I know how lucky I am that people are willing to spend their time, effort, and emotions, and even risk their reputations (however hard we try to anonymize their identities) to increase human knowledge. Their generosity is a gift that should be honored and respected.

Facebook just told us how badly they fucked with that gift.


28 thoughts on “Facebook’s indefensible, unscholarly research [Updated]

  1. I want to know why the reviewers of this paper didn’t flag the ethical problems, and insist those be addressed before publication. The publishers should be ashamed of themselves, let alone the authors.

    • The PNAS doesn’t require that papers go out for external review, from what I can tell. It is up to the assigned editor to make that determination. As for why she didn’t flag it, according to the Atlantic article, she seemed to buy the Facebook argument that since they manipulate their feeds for other reasons, this wasn’t a big deal. But I’m only going by the interview with her in the Atlantic, and I’m hoping she’ll say more about it at some point. Maybe I’m wrong and they did more due diligence than they appear to have. There just isn’t the usual evidence of it.

      It’s an interesting question, i.e., do TOS allowing research to be conducted mean that you consent to any research they want to do? I can’t believe that’s a reasonable interpretation, but these populations are becoming irresistible to researchers (not just those as Facebook). Things are going to get messier before they get clearer, I have a feeling.

      • “The PNAS doesn’t require that papers go out for external review”

        Holy crap. That’s a huge problem right there.

        “do TOS allowing research to be conducted mean that you consent to any research they want to do?”

        I can’t believe any ethical researcher wouldn’t get explicit permission from subjects for *each* experiment. But then, we have proof right here that it’s happening.

        I hope academia pushes back hard against this. No justification at all for it.

        • I could be wrong about the peer review. I’ve seen “may be” and “will be” on two different pages. So I need to hedge on that.

          But yes, otherwise I agree. Even if it did go out, how the hell did this happen?

          ETA: Here are the relevant submission guidelines (the category under which this article was reviewed and accepted for publication:

          Direct Submission.
          The standard mode of transmitting manuscripts is Direct Submission. The Editorial Board screens all incoming submissions and declines more than 50% without additional review. The current acceptance rate of Direct Submissions is 19%. Authors must recommend three appropriate Editorial Board members, three NAS members who are expert in the paper’s scientific area, and five qualified reviewers. The Board may choose someone who is or is not on that list or may reject the paper without further review. Authors are encouraged to indicate why their suggested editors are qualified to handle the paper. A directory of PNAS member editors and their research interests is available at nrc88.nas.edu/pnas_search. The editor may obtain reviews of the paper from at least two qualified reviewers, each from a different institution and not from the authors’ institutions. The PNAS Office will invite the reviewers, secure the reviews, and forward them to the editor. The PNAS Office will also secure any revisions and subsequent reviews. The name of the editor, who is to remain anonymous to the author until the paper is accepted, will be published in PNAS as editor of the article. Direct Submissions are published as “Edited by” the responsible editor and have an identifying footnote.

  2. Great post, Sunita. I’m especially anxious to see how UCSF responds. That the study was constructed in a way to isolate the academics from the data (and thus the approval of their IRB) seems, more than anything, an admission of the ethical problems here. Which, of course, makes it even more troubling.

  3. Sorry, the sentence that starts “That the study was constructed in a way to isolate the academics from the data (and thus the approval of their IRB) seems, more than anything, . . . should read:

    That the study was constructed in a way to isolate the academics from the data (and thus the approval of their IRB) seems, more than anything else,

    • Oh, it gets better. PNAS is a weird journal because it’s the NAS house organ. It used to provide special access to NAS members, which led to some not-good articles. Since 2010 it’s had a different system, but there is still a lot of variation in the output. It has a high ranking in some disciplines, not so much in others. Some scholars report a normal peer-review process, while others, well, don’t.

      It turns out that the editor of this article was also the editor of the (now largely discredited) Girl-Named Hurricane article that was making the rounds a month ago. That is a strange coincidence.

    • Oh Ros, I’m so sorry. I’ve experienced episodes of clinical depression and I share your feelings about ever returning to drug therapy. The one thing that is worth remembering in terms of how you might have been affected is that the substantive effects were very small. That doesn’t mean they weren’t significant in a statistical sense (they appear to have been that), but the actual effect on the users was probably not very noticeable.

      I’m in no way excusing what they did, I think it was abominable to do it without prior consent or post-treatment debriefing. But while the sample is large, it is still a small fraction of FB users, some were in the treatment group and the rest in the control group, and even those in the treatment group are not likely to have been hugely affected.

      • I know that and it helps. But even though the overall effects were small, there may have been some individual users for whom the effects on any given day might have been large enough to have an effect. And, before they did the experiment, they had no notion what those effects could have been. To toy with people’s emotions in that cavalier way is unconscionable. So, I am not sitting here thinking FB is responsible for my depression, but I am scared that whatever they try next could be catastrophic.

        • I know you know that; I was just taking refuge in the probabilities, which tends to be *my* security blanket. ;)

          The other aspect of the study that has only been touched on in reporting is that both positive and negative manipulation had a statistically significant effect, but “neutral” feeds led to less posting by users. This suggests that it is in FB’s interest to ratchet up the emotional content of feeds if they want to increase participation/engagement (which they presumably do, because advertising). So curation could skew in that direction in the future, which will have an effect on users.

          • Yes, I know.

            That’s a really interesting point. I definitely don’t want to be in a place where emotional content is being artificially increased, in either direction.

  4. As a non-academic, I’m perplexed by the idea that any academic thought that any portion of this activity was ethically acceptable, even if they attempted to create a silo.

    Of course, I also think FB and its founder are gigantic hypocrites and data thieves when it comes to privacy and account information/usage, so perhaps my judgment is skewed here.

    • I am pretty sure I was wrong; they didn’t create a silo, they just divided up the research/writing tasks that way. A university IRB (probably Cornell’s) signed off on this, despite the fact that its own FAQ is explicit in requiring informed consent of a type that was (most probably) not obtained here. And no post-treatment debriefing seems to have been done.

      Academics have enthusiastically joined with Facebook to conduct large-scale, invasive experiments. If you search for this particular data scientists’ research at Google Scholar, you see a whole list. It’s very, very seductive to see such a huge database. We’re engaged in web scraping research as well to study other questions (I’m part of a project right now that is looking at collective violence via scraping of news articles).

      After reading all the links I’ve added to the post in my updates, it seems pretty clear to me that the IRB and academic researchers were at least as complicit in designing. conducting, and publishing results of a research project that violate scholarly norms on human subject treatment. So while I share your antipathy toward Facebook, they’re not alone in this. Which makes me very depressed.

  5. I’m not a scientist, but my day job is in academic research administration. While my current position focuses on fiscal compliance and reporting, in previous roles I’ve helped prepare study manuals and the like, so I’ve seen the kind of informed consent documentation that makes it past our university’s IRB board. They’re clearly separated from any other forms the subject is signing off on, they’re written in very simple English with no legalese, and any scientific terminology is defined in lay terms. Heck, the FONT is usually on the large side.

    So while it doesn’t surprise me that Facebook is evil, it offends me that a respected institution like Cornell OK’ed this study. I mean, if the physical therapist I’m seeing to help rehab a sprained ankle were participating in a study trying out a new *sprain therapy*, she would’ve sat down with me, walked me through the consent form, made sure I was clear I’d still receive the best current standard treatment if I refused to participate in the study, and so on. But Facebook manipulated user *mental states* with no meaningful consent, I have no way of knowing if I was one of the subjects of this little study, and Cornell thought this was acceptable? My mind boggles.

    Really, we have enough problems in this country with people distrusting science on issues like vaccines (and medicine in general), global warming, and evolution without an actual breach of trust like this. Argh.

    And yet…I’m still hesitant to give up Facebook altogether. I’d lose touch with too many cousins and high school and college friends. I keep my privacy settings pretty tight and learned years before FB became a big thing to be careful about how much of myself I reveal online, but I’m not happy with the situation.

    • Your last two paragraphs really sum up the dilemma, don’t they? The damage to trust is potentially huge, especially in today’s science-suspicious environment. But Facebook relationships are so valuable to people and to build alternative networks is very costly (not in money necessarily, but in other ways).

      It turns out Cornell did NOT do the IRB review. I’m about to update this post for the 6th time! Time to write a new post, I guess.

  6. Pingback: We Interrupt This Mockery for Some Important Links | A Willful Woman...

  7. I don’t consider an internal Facebook review to be the equivalent of an institutional IRB review. For one thing, as a for profit corporation, FB’s mission is to maximize value for shareholders. That is a fiduciary duty for its officers and directors. With that, no natural expertise in these types of ethics reviews, and the conflicts of interest, I don’t see how it could do an adequate internal review.

  8. The implications of this are terrifying. Not just for the failure of ethics in academic research.

    Anyone who has studied propaganda (its history, or psychology, or political/commercial/social dimensions of its use) will appreciate the dimensions of this “experiment”.

    Social media are not so special that decades of research into social interactions, social psychology, group dynamics, etc are invalidated. Everything I have read to date reinforces my understanding that much of the research around social media technologies suggests that it is the perspective of the observer (and their assumptions) that leads them to misunderstand what is going on – not that social media is some magical new ‘place’ where humans suddenly transform into different creatures.

    Am I wrong?

    Regardless of any thing else – we already knew how effective this kind of behaviour is in manipulating human behaviour – marketing relies on it every day …

    • No, I don’t think you’re wrong, although I’d amend what you’re saying a bit. Much of the work on social interaction and group dynamics applies here.

      There is a growing body of research and writing in academics that addresses the types of manipulation that have historically fallen under marketing and advertising. We’ve treated those as somewhat separate issues from other types of manipulation, but given the sophistication with which Facebook, Google, and other companies are now able to target advertising to users, it is something that needs to be revisited.

      The environment in which information is consumed matters, and social media is a particular type of environment. If you know you’re being advertised to, then you consciously or subconsciously adjust to that. Facebook, though, increasingly modifies and shapes not just the advertising but the non-commercial communications that appear in your feed. To the extent that you don’t think about that happening or don’t have a worked-out way of thinking about it, you are going to be affected differently by those choices. I assume a Facebook user thinks about her friends’ posts and sponsored posts differently. If Facebook is manipulating the latter for reasons of its own, the user isn’t necessarily going to have the same defenses or internal arguments in place against it.

  9. Cornell has withdrawn their claim that Goverment funded the research. At the bottom, it now reads: “Correction: An earlier version of this story reported that the study was funded in part by the James S. McDonnell Foundation and the Army Research Office. In fact, the study received no external funding.”

    • Thanks for posting; I just saw that this morning. It appears that perhaps the video interview in the article discussed more than one project, but I’m not entirely sure. There is so much correcting and amending going on that it’s hard to keep track.

  10. Pingback: Facebook menjao prikazan sadržaj za potrebe psihološke studije

  11. Pingback: Stumbling Over Chaos :: In which Linkity is brief and on a Thursday, confusing everyone

  12. One has to wonder how clever the ladies and gentlemen at Facebook were with creating the fiction that FB’s Data Use Policy covers this experiment.

    Still, what’s interesting is that after a brief Google search (hopefully without receiving manipulated results) I have the feeling that it’s usually the researchers who are being critical …although out there are some 700 000 people who were manipulated. And with this case being the only disclosed, how can we be sure there weren’t other about which no paper was published?

    • My guess is that most of the FB subjects haven’t seen the stories about it. And from what FB research people have written, they’re doing studies all the time, and they don’t seem to be limited to the standard “commercial” tests (e.g., Google’s test of different colors of blue for their logo). The more I learn about it, the less comfortable I am with it.

Comments are closed.