CrowdCamp Report: Just-In-Time Emoji

Emoji provide an expressive platform for non-verbal online communication, and hundreds of emojis are now available for use on smartphones and chat clients.  But the set is by no means complete; many emojis are missing and desired.

At CrowdCamp at HCOMP 2013, we asked, can we create these missing emoji just-in-time by drawing on the crowd? We explored three methods: creating rebuses of existing emoji, finding images on the web, and drawing a new emoji on demand.

Rebus Approach

The first approach asks Mechanical Turk workers to put together a rebus, a sequence of existing emoji and possibly other letters and punctuation.  We gave workers an editor displaying Github’s extended emoji set:

RebusEditor

MTurk workers produced some very interesting and creative results, including:

 

Prompt

Rebus Created by Mechanical Turk Workers Work Time

taco

    TackO 20s

sandwich

 BreadChickenBread 37s

unicorn

 UniCorn 19s

The Dark Knight

 PenDN2DN3DN4DN5 162s

This approach demonstrates that crowd workers are capable of doing a defined just-in-time creative task that requires the mixing and matching different components.

Image Search Approach

The second approach presents an image search of the desired word or phrase to a crowd worker, with all the images rescaled to emoji size, and asks the worker to pick three that best represent the word or phrase:

GoogleImageSearch

We prototyped this task using other HCOMP attendees, and collected the following results:

Prompt

Images filtered by workers Mean work time

unicorn

 UnicornResults 46s (includes reading instructions)

taco

 TacoResults 26s

blonde girl

 BlondeGirlResults 31s

In general, the amount of time to select images was very short, so little creativity was required. The results were strong for some of the simpler keywords (unicorn / taco) but more scattered for an image without a readily discernable emoji (blonde girl).

Drawing-on-Demand Approach

A third approach was to ask the worker to draw an original image using a web-based pixel editor.  Here are some of the results,

Prompt

Image Drawn by Worker Elapsed Time

Unicorn

 UnicornDrawn 30s

Taco

TacoDrawn 30s

Blonde girl

 BlondeDrawn 47s

We Are the World

 WeRWorldDrawn  8m 15s

Discussion

In general, for simple requests, all three approaches produced creative and comprehensible emoji in a short time. Each approach has its own advantages and disadvantages. For example, the rebus approach produces more interesting results but might not produce as good results for a keyword like “Spock”–a character from Star Trek–where the image search approach might be more appropriate.

In general, as the requests got more abstract, such as movie and song names like “Shawshank Redemption” and “We Are the World”, the time to create an answer increased, and comprehensibility dropped.

 

Michael Weingert, University of Waterloo

Rob Miller, MIT

Jenn Thom, Amazon

Pao Siangliulue, Harvard University

Shih-Wen Huang, University of Washington