Use of Mechanical Turk as a MapReduce Framework for Macular OCT Segmentation

Area of Science:

• Ophthalmology
• Medical Imaging
• Computational Science

Background:

• Manual segmentation of retinal layers in spectral domain optical coherence tomography (SD-OCT) is crucial for diagnosing eye diseases.
• Existing methods can be time-consuming and expensive.

Purpose of the Study:

• To assess the feasibility of using Amazon Mechanical Turk for large-scale, parallelized manual segmentation of SD-OCT images.
• To evaluate the cost-effectiveness and reliability of this crowdsourcing approach.

Main Methods:

• A MapReduce framework was employed to distribute SD-OCT image segmentation tasks to Amazon Mechanical Turk workers.
• Workers were instructed to outline retinal sublayers, with each image segmented twice for reliability checks.
• Custom HTML5/JavaScript interface and R for data analysis were utilized.

Main Results:

• Over 93,500 data points were collected for 61 SD-OCT images at a cost of $1.21 per volume.
• High interrater reliability was achieved (Pearson's correlation = 0.995).
• Tasks were completed efficiently by 22 workers, with an average completion time of 4.43 minutes per task.

Conclusions:

• Amazon Mechanical Turk is a viable, cost-effective, and scalable platform for manual SD-OCT image segmentation.
• This crowdsourcing model offers a high-availability solution for generating large datasets for ophthalmic research.