The NPR podcast Planet Money (of which I am a fan) just released an episode that looks at Amazon’s Mechanical Turk (MTurk)- a website where people are paid small sums of money to complete online micro-tasks (like categorising photos or writing transcripts of recordings). The site is named after an 18th Century chess playing ‘robot’ that was actually controlled by a hidden person (see above). This is an elegant analogy for the modern MTurk platform where anonymous workers complete Human Intelligence Tasks (HITs) like a well oiled machine. If you want a slightly more in-depth and accessible introduction then I’d heartily recommend listening to Planet Money‘s short episode (available here).

However, while the episode does provide a useful introduction, as a researcher who has run a variety of studies on MTurk, I also have some specific (/pedantic) reservations with what their portrayal. In some ways the Planet Money reporters went to a lot of effort to dig into the topic: interviewing one of the founders of MTurk and embedding a ‘secret message’ in an MTurk task so that they could speak to some actual workers. Yet  I still found listening to the episode frustrating, due to some surprising misrepresentations that implied a low level of fact checking. I realise what follows will likely be too ‘insider baseball’ for most people, but for those who have some interest in how research on MTurk works, below are some of the biggest inaccuracies/pet peeves I had with the episode. (more…)