All the news that’s fit to read: which social annotations matter

Despite the ubiquity of social annotations online, little is known about their effects on readers, and relative effectiveness.

Despite the ubiquity of social annotations online, little is known about their effects on readers, and relative effectiveness.

Every day, we are influenced by people around us. Increasingly, in addition to receiving news via broadcast media, our friends tell us about news items both on social media and through word of mouth.  The provenance of news items is important.  We use these provenance to determine how interesting a news item is, how much attention we should pay, and how much we should trust it.  As online news reading becomes more social, news websites and applications today show articles recommended by friends, strangers, and computer algorithms: How do people respond to these annotations differently?

To answer this question, we conducted two experiments. The first experiment simulates a user’s logged-out experience with annotations from strangers, a computer agent, and a branded company.  This experiment is intended to shed light on how users respond to annotations when the service provider do not know anything about the user (such as when no social network information is available)

560 participants from Mechanical Turk looked at real news items with the above three types of annotations.  The participants then decided which articles to read based on the headline and these annotations.  Comparing with a control condition with no annotations allowed us to understand the influence of these annotations.  Unsurprisingly, participants clicked just as many news headlines with strangers’ annotations than those with no annotations. Yet, participants clicked on those headlines with annotations from unknown companies (we made up the name of the company) more frequently than the baseline. This suggests strangers’ annotations aren’t persuasive, but those by companies are.

The second experiment simulates an experience with personalized content and annotations from friends (we show articles that a user’s friends have publicly +1’d on Google).  Compared to the baseline (no annotation),  friend annotations are persuasive. Participants clicked more news articles with such annotations, and also rated them as more interesting (than similar articles with no annotation). In contrast, articles that strangers annotated were rated lower than similar articles with no annotation.

Post-experiment interviews indicated possible explanations– friends above a threshold of closeness had known expertise and shared context, for instance. Strangers, on the other hand, lacked these qualities.

Who influences what we read online? Not strangers, but existing friends.  Not only do friends persuade us to read articles that they recommend, they also convince us that these articles are interesting. Surprisingly, branded companies also have a persuasive effect.

For more, see our full paper, All the News that’s Fit to Read: A Study of Social Annotations for News Reading, at CHI 2013. (If you’re attending CHI, this is in the Social Tagging session, Wed 11am).

Chinmay Kulkarni, Stanford University
Ed Chi, Google, Inc.