We studied the behavior of volunteers collecting data for an environmental organization, Close The Doors. They registered whether shops left their doors open or kept them closed during winter, using a mobile app while going about their everyday lives over a two week period. We compared the performance and attitudes of volunteers who scored points displayed on a leaderboard with those who used a control version of the mobile app – still collecting data, but no performance feedback.
We found that:
- The top scorers in the points group substantially outperformed the top scorers in the control group.
- But the lower scorers in the points group performed less well than the lower scorers in the control group.
- Unless additional payment was used alongside points, there was no statistically significant difference in the data collection performance between those awarded points and the control group.
We conducted interviews with top, medium and low scorers in each group to understand what was happening.
- The top scorers were motivated by the leaderboard, competing with those close to them and spurring each other on, This resulted in increased performance. So they performed better than the top scorers in the control group.
- Low scorers were demotivated by the leaderboard, feeling they couldnt catch up and so gave up as the experiment progressed
Our CSCW2014 paper focuses on the attitude of those in the middle. Three of the four mid-scoring interviewees who were interviewed (unlike all but one of the top and low scoring interviewees) did not express competitive attitudes to the leaderboard. Rather, they viewed it as a means of understanding what other volunteers were doing, with the aim of making a typical contribution.
- They were positively motivated to make a contribution on a par with others. One explicitly said they wanted to be in the middle of the leaderboard.
- However the score required to be in the middle is determined by the performances of those below, not by those above.
- So despite the positive motivation, the actual contribution of those in the middle was lower than those in the control group.
So some are motivated pr demotivated by competition, while others are motivated more by playing their part. Corwdsourcing systems could support the latter motivation by using normification in addition to gamification. This is to provide information about the behaviour of others in a way which encourages non-competitive comparison. Perhaps crowdsourcing systems could use adaptive, personalised interfaces to tailor the motivational information they provide based on the psychology of the individual.
For more, see our full paper, Competing or Aiming to be Average? Normification as a means of engaging digital volunteers.