Amazon Mechanical Turk is often considered the original workhorse of all crowdsourcing platforms.
Amazon Mechanical Turk (also referred to as MTurk) is one of the most commonly used crowdsourcing platform among researchers. It was an early player in the field, and for many, it’s still the first place they turn to, particularly for crowdsourcing data collection and labeling. It’s also cost competitive, relying on a pool of almost infinite cheap labor. The platform is designed for crowdsourcing relatively small microtasks (also referred to as HITs or human intelligence tasks) such as labeling images or completing surveys.
Lionbridge is a simple, effective and high-quality alternative to Amazon Mechanical Turk.
Similar to Mechanical Turk, Lionbridge is a solution to get crowdsourced human-annotated data. However, unlike Mechanical Turk, Lionbridge manages the entire process, from designing workflows to sourcing qualified workers. With 500,000+ vetted contributors across 300 languages, large datasets are processed quickly and at high quality. With solution centers in 27 countries, Lionbridge AI supports simple data collection projects as well as complex long-term projects.
Here’s what you can’t do with Mechanical Turk
Full Project Management
While Amazon Mechanical Turk is often considered a cheap solution, there are actually many hidden costs. Requesters are required to build crowdsourcing tasks (called HITs) and source workers from scratch, which can be incredibly time consuming. Good requests require a lot of effort to create – meaning that project managers oftentimes need to invest extra hours into setting up workflows. These limitations make MTurk only feasible for small-scale projects.
One of the key problems in using Mechanical Turk to create training data is the issue of quality. The platform itself offers very little in the way of quality control mechanisms, advanced worker targeting, or detailed reporting. Because MTurk is an open crowdsourcing platform, anybody with a computer and internet connection can sign up and pick up jobs. Recently, there have even been concerns regarding fraudulent workers using bots to mimic human behavior on the platform.
Multilingual Training Data
Although Mechanical Turk is one of the most popular methods to collect data for machine learning, it’s not the right choice for everyone. In fact, it can be difficult to use from outside of the United States. Furthermore, Mechanical Turk generally does not support languages other than English at scale. While there are a number of bilingual workers working on the platform, finding enough qualified native speakers is a difficult task. In contrast, Lionbridge AI provides training data services in over 300 languages and dialects.
|FEATURES||AMAZON MECHANICAL TURK||Lionbridge AI|
|Number of workers||100K-200K untested workers||500K pre-tested contributors|
|Built-in Quality Assurance|