Distributed Sensemaking: Improving Sensemaking by Leveraging the Efforts of Previous Users

Kristie Fisher, University of Washington & Microsoft Research
Scott Counts, Microsoft Research
Aniket Kittur, Carnegie Mellon University

Making sense of digital information one gathers from the web can be a daunting task. Numerous other researchers have explored the process of sensemaking for individuals as well as for groups of people working collaboratively towards a specific goal. In the work we present in our paper, we explore the viability of a distributed sensemaking system to assist users in sensemaking. That is, we tested whether a user’s sensemaking efforts can improve in quality and efficiency when she is able to leverage the sensemaking efforts of others with whom she is not explicitly collaborating with and does not even know.

Put simply, if a few other users have already sifted through some information on a topic and found a way to make sense of it, can having access to the output of this distributed sensemaking save me time and result in higher-quality sensemaking? Alternatively, is it more trouble to try to sort through the sensemaking efforts of others?

We had a few initial users engage in several sensemaking tasks. They were asked to visualize their sensemaking by creating “knowledge maps,” which included links and other web content that they found through online searches. Then subsequent users were asked to engage in the same sensemaking tasks, but each had access to an initial user’s knowledge map to build from. We repeated this cycle several times until we had knowledge maps that had gone through three rounds of iteration (see below). We then asked a separate group of users to engage in the same sensemaking tasks, and in doing so they either had to start a) from scratch, b) from the “knowledge map” of an initial user, or c) from the final “knowledge map” that resulted from the distributed sensemaking process. We compared user sensemaking experience on a number of dimensions among the three conditions. In a separate task, we also tracked users eye movements as users examined knowledge maps in various states of iteration.

Evolution of a knowledge map over several iterations during a distributed sensemaking task

We found that distributed sensemaking is indeed a viable system. In particular we found that:

1) A user is helped more by the framework for organizing information that emerges from distributed sensemaking than by the information itself. For instance, see how a structure for the information emerges after just two iterations in the above example.

2) Having access to the distributed effort of merely four prior users (e.g., “Iteration 3″ above) significantly improved sensemaking quality for subsequent users compared to when they were asked to start from scratch. However, users did not find it helpful to start from one initial user’s work. It is a good thing that only a few users are needed for a “distributed” system to be successful, but it also presents a “startup cost” challenge in that a few initial users have to be willing to invest more time and energy to start the iteration cycle in order for subsequent users to benefit.

For more, see our full paper