A Perspective on Crowdsourcing and Human-in-the-Loop Workflows in Precision Health

Area of Science:

• Artificial Intelligence in Healthcare
• Computational Psychiatry
• Human-Computer Interaction

Background:

• Machine learning models offer high performance for health diagnostics but struggle with complex, high-dimensional, and nonlinear data.
• Overfitting is a significant challenge, particularly for diagnostic systems predicting behavioral and psychiatric conditions with subjective criteria.

Purpose of the Study:

• To explore the emerging field of crowd-powered diagnostic systems.
• To discuss the integration of crowdsourcing into human-in-the-loop machine learning for health diagnostics.
• To identify challenges and opportunities in this nascent area.

Main Methods:

• Leveraging crowdsourcing platforms to gather annotations of complex behavioral features from crowd workers.
• Utilizing these annotations directly for diagnosis or as input for diagnostic machine learning models.
• Integrating crowdsourced data into human-in-the-loop machine learning workflows.

Main Results:

• Crowdsourcing provides a scalable solution to generate labeled data for complex diagnostic tasks.
• Crowd-powered systems can potentially mitigate overfitting issues in machine learning models.
• This approach can enhance the accuracy and efficiency of diagnostic predictions.

Conclusions:

• Crowdsourcing offers a promising avenue to improve machine learning-based diagnostics for nuanced health conditions.
• Careful consideration of crowdsourcing integration can accelerate health screening, diagnostics, and patient access to care.
• The synergy between crowdsourcing and machine learning holds significant potential for advancing computational psychiatry and healthcare.