CrisisLex: Efficiently Collecting and Filtering Tweets in Crises

Timely location of useful information during crises is critical for those forced to make life altering decisions. To stay informed, emergency responders and affected individuals increasingly rely on social media platforms, specifically on Twitter. To obtain relevant information they typically use one of the two main strategies to query Twitter:

crisislex-iswsm2014

  • Keyword-based sampling: Track the tweets that contain a set of manually identified keywords or hashtags specific to a crisis, such as #sandy for Hurricane Sandy or #bostonbombings. Yet, keywords are only as responsive as the humans curating them and, indeed, in our data, such searches returned only a fraction of the relevant tweets—only 18% to 45% of the crisis-relevant tweets were retrieved, with an average of ~33%.
  • Geo-based sampling: Track the tweets that are geo-tagged in the area of the disaster. Alas, by doing so out of the returned tweets only a small percentage are actually about the disaster—only 6% to 26% from the returned tweets are crisis-relevant, with an average of ~12.5%.

Efficiently collecting crisis-relevant information from Twitter is challenging due to the laconic language and the Twitter’s API for accessing tweets in real-time (the streaming API) limitations. Twitter can be queried by content, through the use of up to 400 keywords, or by geo-location. Specifically, if both keywords and geo-locations are given the query is interpreted as a disjunction (logical OR) of both. This is undesirable, as the public API gives access to only 1% of the data, and if the query matches on more data than that, it will return a random sample from it. Thus, as the query becomes more broad, after some point we start losing data.

To overcome these limitations, we built CrisisLex—a lexicon of terms that frequently appear in tweets posted during a variety of crises. By querying Twitter using CrisisLex, we obtain better trade-offs between how much relevant data we retrieve and how clean that data is. The lexicon contains terms such as:

  • damage
  • affected people
  • people displaced
  • donate blood
  • text redcross
  • stay safe
  • crisis deepens
  • evacuated
  • toll raises

CrisisLex has two main applications:

  • Increase the recall in the sampling of crisis-related messages (particularly at the start of the event), without incurring a significant loss in terms of precision.
  • Automatically learn the terms used to describe a new crisis and adapt the query with them.

Consequently, CrisisLex requires no manual intervention to define or adapt the query. This is particular useful, as the manual identification of keywords requires time which, in turn, may result in losing tweets due to latency. In addition, using CrisisLex does not only retrieve more comprehensive sets of crisis-relevant tweets, but it also helps to preserve the original distribution of message types and message sources.

For more detailed results on how we build and tested CrisisLex please check our paper: CrisisLex: A Lexicon for Collecting and Filtering Microblogged Communications in Crises. If you want to use CrisisLex to collect tweets, and/or want to build your own lexicon for other domains (e.g., health, politics, sports) please check our code and data (in accordance with the terms of service of Twitter’s API) at CrisisLex.org

Alexandra Olteanu, École Polytechnique Fédérale de Lausanne
Carlos Castillo, Qatar Computing Research Institute
Fernando Diaz, Microsoft Research
Sarah Vieweg, Qatar Computing Research Institute

Improving recommendation by directing the crowd’s attention

We are drowning in content. On YouTube alone, over 100 hours of video are uploaded every minute. Which of them are worth watching? Which of the thousands of news stories and discussions on Reddit are worth reading? Which Kickstarter projects are worth funding? To identify quality items, content providers aggregate opinions of many, for example by asking people to recommend interesting items, and prominently feature highly-rated content. In practice, however, peer recommendation often creates “winner-take-all” and “irrational herding” behaviors with inconsistent, biased and unpredictable outcomes in which items of similar quality end up with wildly different ratings.

Researchers from USC Information Sciences Institute and Institute for Molecular Manufacturing demonstrated that it is possible to overcome these limitations to improve the ability of crowds to identify interesting content. Due to human cognitive biases, people pay far more attention to items appearing at the top of a web page than those in lower positions. Hence, the presentation order strongly affects how people allocate attention to the available content. Using Amazon Mechanical Turk, researchers demonstrated that they can manipulate the crowd’s attention through the presentation order of items to improve peer recommendation. Specifically, the common strategy of ordering items by ratings does not accurately estimate their quality, since small early differences in ratings become amplified as people focus attention on the same set of highest-rated items.  This “rich-get-richer” effect occurs even when the ratings are not explicitly shown, but are simply used to order the items.

In contrast, ordering items by the recency of rating, much like a Twitter stream with the most recently retweeted posts at the top of the stream, leads to more robust estimates of their underlying quality and also produces less variable, more predictable outcomes. Ordering items by recency of rating is also a good choice for time critical domains, where novelty is a factor, since continuously moving items to the top of the list can rapidly bring newer items to crowd’s attention.

PlosOne-fig

By judiciously exposing information about the preferences of others, for example, by changing the presentation order, content providers can better leverage the “wisdom of crowds” to accelerate the discovery of quality content.

Lerman K, Hogg T (2014) Leveraging Position Bias to Improve Peer Recommendation. PLoS ONE 9(6): e98914. doi:10.1371/journal.pone.0098914

 

Kristina Lerman, USC Information Sciences Institute

Tad Hogg, Institute for Molecular Manufacturing

Posted in -

How Your Digital Footprints Reveal the Events You will Want to Attend

What drives the choice of social media users to attend certain events rather than others? The answer to this question finds vital applications in personalized event recommendations. Yet, it has only been recently that the avalanche of user generated content from location-based social services truly allows the exploration of this aspect at a relevant scale.

User check-in activity heatmap of London in the days before (left) and during (middle) the UEFA Champions League Final. Darker shaded regions denote a higher number of check-ins closer to the observed maximum among all regions during the same day. The size of the location markers in the rightmost figure is proportional to the number of check-ins at the place. Notice the significantly increased activity at the Wembley area in Northwestern London on the 28th of May 2011 when the UEFA football match was held (middle, right).

User check-in activity heatmap of London in the days before (left) and during (middle) the UEFA Champions League Final. Darker shaded regions denote a higher number of check-ins closer to the observed maximum among all regions during the same day. Notice the significantly increased activity at the Wembley area in Northwestern London on the 28th of May 2011 when the UEFA football match was held (middle, right).

In this work we take advantage of the location broadcasts of Foursquare users to study the social and behavioral underpinnings of event participation in three metropolitan cities – London, New York and Chicago. The main challenge we address is: what is the extent to which temporal, spatial, and social factors influence user’s decision to visit one event over another?

Word clouds of the words used in the names of the places and place types for several events where Foursquare users check in: (a)-(b) London, (c)-(d) New York, (e)-(f) Chicago

Word clouds of the words used in the names of the places and place types for several events where Foursquare users check in: (a)-(b) London, (c)-(d) New York, (e)-(f) Chicago.

Not surprisingly, we confirm that social factors in their various manifestations are the dominant players when it comes to event preferences.

  • Event popularity, which can be related to forces of social contagion, dominates the factors in London. We find that there are a few massively popular events in cities such as the Royal Wedding in London where more participants are lured to the crowd by forces reminiscent of gregariousness and preferential attachment.
  • An explicit social filtering that checks whether friends are visiting the event tops the results in New York and Chicago. This complementary finding highlights even more the social nature of events and the gravitational aspect of friendship. If your friends are attending an event, with a high likelihood you will be joining them as a part of a social group.
  • The friends’ visited place types such as bars, theaters or stadiums and the associated activities with them are also indicative of the users’ event preferences. We model this assumption by computing attraction scores towards events in a socio-spatial graph that connects users, place types and events. The modelling proves especially suitable to recommend niche content, i.e. events which are more appealing to a specific group rather than the general audience.

For more, see our full paper, The Call of the Crowd: Event Participation in Location-based Social Services.
Petko Georgiev, University of Cambridge
Anastasios Noulas, University of Cambridge
Cecilia Mascolo, University of Cambridge

The good, the nerd or the beautiful: who should I choose to work with me?

During our lives, we perform collaborative tasks in a wide and diverse range of activities, such as selecting students to participate in a school project, hiring employees to a company or picking up players for a football friendly match.

Given this context, we ask: what factors influence such decisions, i.e., what factors are determinant for selecting/repelling someone for a given collaborative task?

motivation

Without much thought, one could answer this fundamental question by saying that the skill of a person to do the task determines if she/he will be selected for a collaboration. Although we agree proficiency definitely plays an important role in the decision, we again ask: is proficiency the only determinant factor? If not, is proficiency even the main factor?

From a very careful an particular experiment conducted in a classroom of undergrad students, we mixed data from an offline questionnaire with Facebook data to reveal a number of interesting and sometimes surprising findings:

  • the most skilled students were not always preferred;

  • a number of social features extracted from Facebook (see table bellow), such as the strength of the friendship, the popularity of the individual on Facebook, if she is extrovert, and her similarity with other students, are more informative than the grades to determine the willingness of students to work together.

features

Our findings show:

  • the importance of building up a wide and diverse personal profile when the aim is to be selected for a given collaborative task;

  • that online social network data can indicate if two individuals would like or not to work together and, as it is well know, social chemistry is desirable for achieving maximum performance of a team;

  • a potential to leverage several online applications, such as team and collaboration recommendation systems that highlight potential fruitful collaborations and hide collaborations between potential conflictual relationships.

Douglas D. Castilho, Universidade Federal de Minas Gerais, Brazil

Pedro O.S. Vaz de Melo, Universidade Federal de Minas Gerais, Brazil

Daniele Quercia, Yahoo! Labs, Barcelona

Fabricio Benevenuto, Universidade Federal de Minas Gerais, Brazil

The Tweets They are a-Changin’: Evolution of Twitter Users and Behavior

Image

Over the years, we have seen significant amounts of research on Twitter, due to the ease of access to large amounts of data. However, most studies typically focus on data from small period of time, generally ranging from a few weeks to a few months. Given that Twitter has evolved significantly since its founding in 2006, this situation makes it hard to interpret prior results or make projections of where Twitter is headed.

Our work aims to quantify the evolution of Twitter itself, focusing on the public Twitter ecosystem. There are two main contributions of our work: First, we collect a dataset of over 37 billion tweets spanning over seven years. Second, we quantify how the users, their behavior, and the site as a whole have evolved. Below, we highlight a few of our results; the paper contains many more results as well as details on the datasets that we use.

final_users_account

  • While Twitter has grown significantly, it has also seen a large number of users leave the platform. Today, we see that almost 33% of the user population is inactive, over 6% has been suspended, and 2% of users have deleted their accounts.

final_geo

  • We observe Twitter spreading over the globe; the fraction of tweets from the U.S. and Canada has dropped from over 80% to 32% today. Additionally, there has been a massive increase in the diversity of languages used on the platform. The figure above shows this evolution for both user-provided locations and tweet geo-tags.

final_tweet_content

  • We can quantify the rise of malicious activity on Twitter, including both follower spam (we see a massive increase in follower counts in 2011 and 2012) and trending-topic hashtag spam (we see a spike in tweets with many hashtags in 2009).

final_tweet_type

  • We can observe users quickly adopting platform enhancements by Twitter. Before Twitter introduced native retweets, only 5% of tweets were retweets; today, it is over 27%.

final_source

  • Twitter has shifted from a primarily-mobile system (based on SMS) to a primarily-desktop system (based on the web site) and back to a primarily-mobile system (based on smartphone apps). Today, over 50% of tweets come from mobile devices.

We hope that our findings will help researchers to better understand the Twitter platform and to more clearly interpret prior results. We make all of our analysis available to the research community (to the extent allowed by Twitter’s Terms of Service) at http://twitter-research.ccs.neu.edu/.

Yabing Liu, Northeastern University
Chloe Kliman-Silver, Brown University
Alan Mislove, Northeastern University

Who is dating whom: User behavior analysis and prediction for online dating sites

Online dating sites have become popular platforms for people to look for potential romantic partners. 40 million out of 50 million single people in the US have signed up with various online dating sites, and 20% of currently committed romantic relationships began online, more than through any means other than meeting through friends.

Inter-city communications of the online dating site within China

Inter-city communications of the online dating site within China

Few studies have been attempted to understand user behavior on online dating sites. In a previous work, we examined user’s online dating behavior based on a large dataset obtained from a major heterosexual online dating site in China, beihe.com. The main findings of our analysis are described as follows:

  • Many results on user messaging behavior align with notions in social and evolutionary psychology: males tend to look for younger females, while females place more emphasis on socioeconomic status, such as the income and education level.
  • Geographic distance between two users plays an important role. Out of all messages, 46.5% of them are within the same city and inter-city communications (shown in Figure 1) quickly decrease with distance.
  • Profile photos affect male and female’s behavior differently. Females with a larger number of photos are more likely to invite messages and secure replies from males, but the photo count of a male does not have as significant effect in attracting contacts and replies.
  • There is significant discrepancy between a user’s stated dating preference and his/her actual online dating behavior. Users tend to be more flexible than their online write-ups suggest.

More results can be found in our paper Who is Dating Whom: Characterizing User Behaviors of a Large Online Dating Site.

Our ICWSM 2014 paper extends our previous work to predict user’s reply behavior using a machine learning framework. Based on the profiles of the sender and receiver, as well as their prior communication traces, we seek to accurately predict whether the receiver will reply to initial contact messages from the sender, as illustrated in Figure 2. The ultimate goal is to build a reciprocal commendation system that would match users with mutual interest in each other.

Online dating interactions can be modeled as a bipartite network

Online dating interactions can be modeled as a bipartite network

We model our problem as a link prediction problem, which aims to uncover the hidden links of network. Our previous results indicate that user-based attributes are helpful to the prediction model, as a user’s message reply behavior exhibits different correlations with various user attributes including age, income, education level, geographic location, photo count, and etc. In addition to these user-based features, we also extract graph-based features from the bipartite network derived from user communication traces. A user’s message sending and replying rates reflect how actively he/she is looking for potential dates, and the message receiving rate is a good measure of the user’s popularity level. Further, we extract the interactions between users who share similar interest and attractiveness with the sender and receiver and define a set of neighbor-based features appropriate for our bipartite dating network.

Finally, we apply these features with different classification algorithms to predict whether the receiver will reply to a sender. Below are the main findings of our prediction model:

  • User-based features and graph-based features result in similar performance, and can be used for effective user reply prediction. Only a small performance gain is achieved when both feature sets are used.
  • The best performance is achieved by the random forest algorithm with precision and recall rate around 75%.
  • In user-based features, females are most concerned about age, income, house, children and parent status, which males do not show clear trend.

These studies can provide valuable guidelines to the design of recommendation engine that can match users with mutual interest in each other. More interesting results can be found in our full paper: Predicting User Replying Behavior on a Large Online Dating Site.

Peng Xia, University of Massachusetts Lowell
Hua Jiang, Baihe.com
Xiaodong Wang, Baihe.com
Cindy Chen, University of Massachusetts Lowell
Benyuan Liu, University of Massachusetts Lowell

User-created groups in health forums

Health forums are one of the popular sources of health information online. In many health forums, patients and their family members discuss their health concerns via questions and responses on topic-specific user groups. Some forums also allow users to initiate groups on topics of their interest. We wanted to investigate these user-created groups and find out how they differ from site-defined groups.

Based on a comprehensive dataset collected from MedHelp.org, a leading health discussion forum, we found some interesting behaviors in groups initiated by users:

Class distribution of SDGs and UCGs

Category distribution of site-defined groups (SDGs) and user-created groups (UCGs).

  1. Users initiate homophily-driven groups: While site-defined groups primarily focus on specific conditions and treatment options, user-created groups focus on:
    • rare conditions or complications that are not sufficiently covered by the site-defined groups; for example, women experiencing complications after a tubal ligation surgery
    • building and maintaining friendships: for example, pregnant women forming groups to connect with others at the same stage of pregnancy; ADHD teens forming groups to help them socialize
    • demographic-specific topics: for example, groups exclusively for teen mothers, Michiganders suffering from Autism disorders, or women over a certain age who are trying to conceive
    • non-medical, social activities: for example, prayer groups, groups on specific hobbies such as creative writing, etc.
  2. User-created groups tend to be smaller, but more social, more active, and lead to denser friendship links than site-created groups.
  3. Although site-defined groups tend to have more discussion threads, they also tend to get lower number of responses. User-created groups tend to have longer discussions, with more members participating in such discussions.

Further, we find that membership in user-created groups makes users more active, more responsive, and better (closely connected) communities, when compared to users who are only members of site-defined groups. These findings suggest that allowing users to create groups, and providing them with tools to search and join user-created groups would further strengthen the community-building aspects of health forums.

To read about it in more detail, see our full paper, User-created groups in health forums: What makes them special? We’ll be happy to hear from you.

V.G.Vinod Vydiswaran, School of Information, University of Michigan
Yang Liu, School of Information, University of Michigan
Kai Zheng, School of Public Health, Department of Health Management and Policy and School of Information, University of Michigan
David Hanauer, Department of Pediatrics and School of Information, University of Michigan
Qiaozhu Mei, School of Information and Department of Electrical Engineering and Computer Science, University of Michigan

What Foursquare Check-ins Foretell about your Business during the Olympics

Why do some businesses enjoy increased popularity during large scale events whereas others decrease the attention they receive from customers? Thanks to the deluge of user generated content from location-based services such as Foursquare we can now harness publicly available crowdsourced knowledge to gain scalable insights. We can evaluate both geographic and mobility factors by using the location broadcasts of Foursquare users as a proxy to the amount of customers a retail facility sees.

London Transitions

User transitions towards the Olympic park in Stratford, London in the three-week periods before (left) and during (right) the Games. We observe an almost 10-fold increase in the check-in activity during the Olympics compared to the previous period.

We focus on the most recent London Olympic Games to find an answer to the following question: What are the factors determining whether local food businesses close to the Olympic venues and live broadcasting sites will experience a rise in potential customers during the sporting event?

  • The geographic proximity to a sports stadium is one determiner of success. Olympic spectators in need of refreshments may opportunistically visit local food retailers.
  • The diversity of activity in the nearby area. Being positioned in a neighborhood where onlookers have some variety in what they can do is beneficial, especially when they stay longer in the Olympic areas to watch more of the games.
  • The neighborhood sociability. Areas which have proven to be popular among groups of friends tend to benefit more the retailers during the Olympics. Among other things sports games are social events.
  • The likelihood of user transitions from and to entertainment places. If your business has been successful so far in attracting customers from nearby recreation facilities, it will probably be even more so during a large scale event when user activity is sky-rocketing.

What about the historical popularity of businesses? The truth is, in a major event such as the Olympics, when the activity pulse of the city is turned upside down, historical popularity is a weak predictor. Large scale events can act as game changers on the commercial landscape of a city. Places that have been less popular in the past are provided with a novel opportunity to attract new customer flows.

For more, see our full paper, Where Businesses Thrive: Predicting the Impact of the Olympic Games on Local Retailers through Location-based Services Data.
Petko Georgiev, University of Cambridge
Anastasios Noulas, University of Cambridge
Cecilia Mascolo, University of Cambridge

Early adopters of Twitter and Google+: Validation of a theoretical model of early adopter personality and social network site influence

The widespread adoption of social media is transforming the consumer-brand relationship. Social media is allowing consumers connect with other users, create, consume and control access to content (Hoffman and Novak, 2012). Research suggests that social media increases brand relationship depth and loyalty, and generates incremental purchase behaviour (Laroche et al., 2012; Kim and Ko, 2012; Pooja et al., 2012). It is not surprising therefore that commentators suggest that marketers should target social media users who are more likely to exert an influence on their network in order to facilitate brand recommendations (Iyengar, Han, & Gupta, 2009). But who are these influentials? Goldenberg et al. (2009) suggest that there are only two types of influential that impact information diffusion – innovators and followers.

influence

Our study looks at early users or in Goldenberg at al.’s terminology, innovators, of two social networking sites, Twitter and Google+, and the effects of personality and mode of information sharing on social influence scoring. Specifically, we look at:

1. How does (i) extraversion, (ii) openness and (iii) conscientiousness influence:

  • Information sharing behaviour
  • Rumour sharing behavior

2. How does (i) information sharing behaviour and (ii) rumour sharing behaviour impact social network site influence scores?

Early Twitter users were identified through a public list and through the joining date listed on user public profiles. As the study occurred during the Google+ closed field test, all users were deemed early users. Two discrete survey instruments were designed, one for Twitter and one for Google+ to provide for different SNS validation checks. To assess the personality traits of respondents, we tested extraversion, openness and conscientiousness with the scale of Gosling et al. (2003) while information and rumour sharing scale were extracted from Marett and Joshi (2009). The SNS score was the independent variable in our model and this was measured using two commercial SNS influence score providers, PeerIndex and Klout.

Our study hypothesized that that Extraversion and Openness were two personality traits that should positively influence both Information and Rumor sharing behavior (H1 and H2), while Conscientiousness would have a reverse effect on Information (+) and Rumor (-) sharing behavior (H3 and H4). We also hypothesized that both Information and Rumor sharing behavior should positively influence social network influence scoring. A structural equation model using AMOS was used to test these hypotheses.

Results of Structured Equation Model - Standardised Regression Weights and Summary Findings

Results of Structured Equation Model – Standardised Regression Weights and Summary Findings

 

The model suggests:

  • Early users of social network sites who are more extrovert or more open or more conscientious are more likely to share information
  • Information sharing and rumor sharing should be treated as two distinct constructs in the discussion of social network influence.
  • All three traits were negatively related to rumor sharing. Only the effects of extroversion and conscientiousness were significant.
  • Both information sharing and rumor sharing impacted positively and significantly on social network site influence scores.

While previous literature has suggested that it is difficult to identify market mavens (Goldsmith et al., 2006), early users of social media can be identified easily and conveniently. This may provide firms with the opportunity to target potential innovators and early adopters much more efficiently and thus accelerate diffusion of marketing messages. Our study suggests filtering these adopters by messaging behaviour may also be of assistance with a greater of emphasis of resources being placed on those social network users who share information rather than rumor. While identifying these potential influencers would seem to be more efficient than identifying mavens, further research is required to understand the most effective way and time to engage with them. Finally, it would seem social network influence scores provide useful signals for identifying social media users likely to share information. Social media users characterised by a combination of high influence scores and propensity for information sharing are powerful assets for firms, particularly if they have relatively large social networks. Engaging with these influencers represents a relatively low cost mechanism for indirectly reaching target markets through word of mouth on social networks.

The research was conducted by Dr Theo Lynn (DCU Business School), Dr Laurent Muzellec (UCD), Dr Barbara Caemerrer (ESSCA), Prof. Darach Turley (DCU Business School) and Bettina Wuerdinger (DCU Business School).

A More Paradoxical Paradox

Have you ever checked your Facebook and Instagram and felt that your friends have more interesting lives? You’re not alone! In fact, that’s one of the consequences of Friendship Paradox, which states that on average, your friends have more friends than you do. Recently, researchers demonstrated that network paradoxes hold not only for popularity, but other traits as well, such as activity and virality of content received.

Beach

A variety of paradoxes exist in online social network such as Twitter and Facebook: Your friends, on average, have more friends, are more active, and post more popular/interesting content compared to you. Image source: https://flic.kr/p/5QXd9M

We recently showed that the standard version of the paradox, using the mean of friends’ values of the trait, arises trivially from the properties of statistical sampling from a heavy-tailed distribution. Social traits, such as popularity or activity (e.g., number of posts made), often have a “heavy tail”, where extremely large values, e.g., very popular people, appear much more frequently than expected compared to a normal distribution. When sampling randomly from such a distribution, the mean of the sample (i.e., mean of friends’ values) will grow with sample size, resulting in paradox. In contrast, the median of the sample does not behave this way and is a more robust measure of the paradox.

Surprisingly, paradoxes persist when median is used: i.e., most of your friends (and followers) have more friends (followers) than you do, and also post and receive more viral and diverse content than you do. In other words, the paradox holds not only for the mean, where a single very popular (or active) friend could skew the average, but also for most friends.

Why do strong paradoxes exist in networks? Since they are not a consequence of sampling, they must have behavioral origin. We hypothesize that they arise due to correlations between individual’s traits and popularity or between traits of connected people (homophily). To test this hypothesis, we performed the shuffle test: we kept the network topology fixed, but permuted traits between nodes in the network. This keeps the distribution of the traits intact, but destroys correlations between people. As expected, we still observe a paradox for the mean in the shuffled network, but not the strong paradox that uses the median.

In short, main findings of our work are

  • We found “strong” paradoxes where most of your friends have more friends than you do, etc.
  • We showed that the paradoxes have a behavioral origin, and not simply the result of statistical properties of sampling from the network.
  • The origin of the paradoxes is in the correlations between traits of nodes and their degree or homophily.

For details, please see our paper “Network Weirdness: Exploring the Origins of Network Paradoxes” http://arxiv.org/abs/1403.7242

Farshad Kooti, University of Southern California
Nathan O. Hodas, USC Information Sciences Institute
Kristina Lerman, USC Information Sciences Institute