NICE: Social translucence through UI intervention

Aaron Halfaker, University of Minnesota
Aniket Kittur, Carnegie Mellon University
John Riedl, University of Minnesota

Social production systems such as Wikipedia rely on attracting and motivating volunteer contributions to be successful. One strong demotivating factor can be when an editor’s
work is discarded, or “reverted”, by others.  In this research we demonstrate evidence of this demotivating effect and design a novel interface aimed at improving communication between the reverting and reverted editors. We deployed the interface in a controlled field study on the live Wikipedia site, and report on changes in the behavior of 487 contributors who were reverted by editors using our interface. Our results suggest that simple interface modifications (such as informing Wikipedians that the editor they are reverting is a newcomer) can have substantial positive effects in protecting against contribution loss in newcomers and improving the quality of work done by more experienced contributors.

Wikipedia is an interesting case of volunteer participation.  The site maintains a large base of users who voluntarily do all of the dirty work of writing an encyclopedia.  Despite the open and voluntary nature of the encyclopedia, it is of exceptional quality (Giles, 2005).

Recently, there has been press about a decline in the user base and the decreasing retention of new editors has been implicated as the primary cause (Editor Trends, 11)(Suh, 09).

Previous research by Zhang & Zhu (Zhang, 07) showed that a decrease in editor activity could be measured after the articles they created were changed by other editors.  They also found this effect to be more pronounced for newcomers.  They concluded that the decrease in activity represented a decrease in motivation following the implied feedback of having an article changed by someone else.

We suspected that reverts, an edit to an article that discards the changes made by an editor, could represent a much more direct and effective feedback event and that the effects would be similarly deleterious for newcomers.  Motivated by the recent decline in newcomer retention, we designed an experiment to determine the effects of negative feedback could be lessened

Based off of previous work in Social Translucence (Erikson, 00) we hypothesized that editors would interact more effectively with newcomers if they were simply more aware of when they were interacting with one, so we tried something simple: we modified Wikipedia’s interface to show editors when they were reverting newcomers for a group of editors who volunteered to participate.

After analyzing their activity and the activity of the reverted editors, we found a few interesting effects.

  • Newcomers reverted by editors who used the NICE interface were substantially more likely to continue editing
  • Non-newcomers reverted by editors using the NICE interface were substantially more likely to improve the quality of their work

The sheer simplicity of the interface modification and the substantially effect that it had on retention suggests (1) that being reverted can be unnecessarily demotivating for newcomers and (2) improving the situation substantially can be simple and straightforward.

For more, see our full paper, NICE: Social translucence through UI intervention.

References

  1. Bongwon Suh, Gregorio Convertino, Ed H. Chi, and Peter Pirolli., The Singularity if not Near: slowing growth of Wikipedia WikiSym’09.
  2. Editor Trends Study, Wikimedia Foundation, 2011.
  3. Giles, J. Internet encyclopedias go head to head. Nature, 438 (2005), 900–901.
  4. Thomas Erickson and Wendy A. Kellogg. 2000. Social translucence: an approach to designing systems that support social processesACM Trans. Comput.-Hum. Interact. 7, 1 (March 2000), 59-83.
  5. Zhang, X. and Zhu, F., Intrinsic motivation of open content contributors: The case of WikipediaWISE’06.

3 thoughts on “NICE: Social translucence through UI intervention

  1. This is really fascinating work!

    The study is important because it helps motivate contributions to a public good and also methodologically interested.

    I definitely believe that there should be many more field experiments and other natural experiments done in Wikipedia and other online communities. I’d be curious to hear what other field experiments you might be planning.

    • Thanks for the reply. I’ve been designing on new UI changes to field test in wiki and I’ve been working with the foundation to improve the framework for performing these types of experiments too.

      Wikignome is one of my most recent creations. It’s still fresh out of version control, but I’d be very happy to have some feedback if you are interested in giving it a try.

  2. This is really fascinating work!

    The study is important because it helps motivate contributions to a public good and also methodologically interesting.

    I definitely believe that there should be many more field experiments and other natural experiments done in Wikipedia and other online communities. I’d be curious to hear what other field experiments you might be planning.

Comments are closed.