Precision Telemedicine through Crowdsourced Machine Learning: Testing Variability of Crowd Workers for Video-Based...

Area of Science:

• Digital Health
• Precision Medicine
• Artificial Intelligence in Healthcare

Background:

• Telemedicine is crucial for precision health and medicine.
• Crowdsourcing platforms offer a pool of virtual workers for data annotation tasks.

Purpose of the Study:

• To assess the capability of crowdsourced virtual workers in diagnosing autism.
• To evaluate the potential of using virtual workers for behavioral ratings of children.

Main Methods:

• Workers from Amazon Mechanical Turk (MTurk) and Microworkers (Bangladesh, Kenya, Philippines) rated YouTube videos of children.
• Worker responses were inputted into a machine learning classifier trained on clinical data.

Main Results:

• Worker performance varied, but the best responses achieved high diagnostic accuracy (mean probability >80%).
• Accuracy and variability were comparable to expert performance in prior studies.
• A weak correlation was found between time spent and performance (r = 0.358, p = 0.005).

Conclusions:

• Crowdsourced workers can contribute to accurate autism diagnoses.
• Differences exist in crowdworker ability to rate behavioral features, necessitating strategic recruitment.
• This method presents a scalable and affordable approach for precision medicine in pediatric behavioral health.