Mechanical Turk Workers Are Not Anonymous

Users of Amazon Mechanical Turk generally believe that the workers are anonymous, identified only by a long obscure identifier like A3IZSXSSGW80FN. (That’s mine.) A worker’s name or contact information can’t be discovered unless the worker chooses to provide it.

But it isn’t true. Many MTurk workers are rather easy to identify.

Take a typical worker ID. If you’ve ever used MTurk yourself, you can find your own worker ID on your Dashboard, on the far right:

Screen Shot 2013-02-28 at 10.06.03 PM

Just search the web for your worker ID, and you may find a surprising number of results:

  • wish lists
  • book reviews
  • tagged Amazon products

In fact, many workers even have a public Amazon profile page containing their real name and sometimes even a photo, at http://www.amazon.com/gp/pdp/profile/workerID. Here’s my profile:

Amazon public profile showing my real name

In preliminary testing with published datasets containing turker IDs, about 50% of worker IDs we tried had a public profile page, and about 30% of IDs had a discoverable real name.  A smaller percentage had a photo as well.

The fundamental problem here is that all of Amazon uses the same identifier for the worker’s account. Every interaction with Amazon is tagged by that identifier, and many of those interactions produce public pages containing the identifier, which are indexed by search engines.

We discovered this fact at a CrowdCamp workshop last weekend at the CSCW 2013 conference, and it came as a stunning surprise to a room full of researchers with years of experience using Mechanical Turk.

The implications are sweeping:

* For academic researchers: worker IDs may have to be treated as personally identifiable information. For example, publishing worker IDs online in public data sets may be a violation of worker privacy, and counter to the requirements of the researcher’s institutional review board.

* For workers: if you want to protect your online identity and retain anonymity on MTurk, you should register a different Amazon account expressly for MTurk, and not use the same account that you use for Amazon purchasing. But note that if you already have an MTurk account, creating a new one would lose any reputation you’ve built up.

* For Amazon itself: this is a privacy hole that needs addressing. The best solution would be to use distinct identifiers for MTurk and other Amazon properties (even if the same login account). At a minimum, however, workers and requesters should be made aware of this privacy risk, and workers whose accounts are publicly-identifiable should be permitted to create new ones without loss of reputation.

For more detail, see our working paper, Mechanical Turk Is Not Anonymous, which will be posted by March 7.

Saeideh Bakshi, Georgia Tech
Michael Bernstein, Stanford University
Jeff Bigham, University of Rochester
Jessica Hullman, University of Michigan
Juho Kim, MIT CSAIL
Walter Lasecki, University of Rochester
Matt Lease, University of Texas Austin
Rob Miller, MIT CSAIL
Tanushree Mitra, Georgia Tech

 

This entry was posted in CrowdCamp 2013 by Rob Miller. Bookmark the permalink.

About Rob Miller

Rob Miller is an associate professor of computer science at MIT, and associate director of the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). His research interests lie at the intersection of programming and human computer interaction: making programming easier for end-users (web end-user programming), making it more productive for professionals (HCI for software developers), and making humans part of the programming system itself (crowd computing and human computation).

9 thoughts on “Mechanical Turk Workers Are Not Anonymous

  1. Turkers have known about that for some time. This post on a forum from last year shows that turkers are already aware. Sorry you will need an account on the forum to be able to read it at the link.

    http://turkernation.com/showthread.php?16093-Requester-asking-for-my-MTurk-ID&p=178573&viewfull=1#post178573
    “Generally it’s fine to give out your worker ID, a lot of requesters will ask for it. Just be aware that your worker ID can be linked back to your Amazon profile, if you have set one up — for instance, if your worker ID is A2KEKKJ9CAC2KC, http://www.amazon.com/gp/pdp/profile/A2KEKKJ9CAC2KC will tell the world about you, if you want it to. So you probably shouldn’t use any of the Amazon profile features on the Amazon account you turk with.”

    Many have killed their Amazon profile because of it. But, do a Google search of a worker ID and it will show up in reviews of products on Amazon. The only defense we have is to kill our Amazon profile and to never use the mturk account to post a product review on Amazon.

  2. Of course it would take the combined insight of a clowder of crowd researchers to notice this!

    Assuming that Amazon doesn’t take swift action on this, do you think that there are additional things that researchers need to do other than avoid publishing ID’s in order to protect workers?

    Are there advantages? Panos mentioned this paper on Twitter where they used this fact to review worker history: http://www.ipeirotis.com/research/publications/bonus-disclosure-and-choice-what-motivates-the-creation-of-high-quality-paid-reviews

  3. Pingback: CrowdCamp Report: Benchmarking the Crowd | Follow the Crowd

  4. I noticed this a while ago while shopping on Amazon; it identified me in the URL with what I know as my MTurk ID, although I guess it’s just my Amazon ID. My profile is super-private (initials only, no pictures), but I have left reviews on items in an attempt to be helpful, and I’m sure that some of those have revealed information about me. I’m bummed out but I doubt that Amazon will ever do anything, though.

  5. There is some dangerous advice in this post. “For workers: if you want to protect your online identity and retain anonymity on MTurk, you should register a different Amazon account expressly for MTurk,”. Creating multiple accounts is a violation of Amazon’s TOS and will result in the worker being permanently banned from MTurk.

    • It’s true that Amazon forbids one person from using multiple accounts on *MTurk*. But having one account for your MTurk work and a different account for your Amazon purchasing is just fine. An earlier version of this blog post had exactly the warning you raised, but when we talked to Amazon about it, they reassured us that it’s acceptable to do this.

  6. Pingback: Weekendowa Lektura | Zaufana Trzecia Strona

  7. COME ON GUYS, YOU CAN CHANGE SETTINGS TO “PRIVATE”.

    MY WISH LIST, EMAIL ADDRESS, ETC. ARE PRIVATE.

    ONLY THING PEOPLE SEE ARE MY PEN NAME AND NICK NAME, WHICH I CHANGED TO “MTURKER AND TURKMAZING”.

    Sorry for the all-caps, just seems like you guys missed that.

    Of course, people need to be aware of that, or else they might put quite a bit of information out there.

  8. Pingback: That Money Thing | ericality

Comments are closed.