Crowdsourcing and machine learning contests in Parkinson's disease research - when do they work?

Area of Science:

• Neuroscience
• Computational Biology
• Artificial Intelligence

Background:

• Crowdsourcing and machine learning contests are emerging tools in Parkinson's disease (PD) research.
• These methods offer novel approaches to analyze complex datasets and solve challenging problems in neurodegenerative disease research.

Purpose of the Study:

• To review the use of crowdsourcing and machine learning contests in PD research.
• To identify best practices for designing successful contests.
• To explore future opportunities for these methodologies in PD research.

Main Methods:

• Literature survey of crowdsourcing applications in PD research.
• Comparative case study of two machine learning contests: MJFF Freezing of Gait (FOG) Challenge and AMP PD Proteomics Challenge.
• Development of a taxonomy for crowdsourcing projects and a framework for contest success characteristics.

Main Results:

• Contest success is highly dependent on design factors, particularly problem solvability and metric selection.
• The FOG Challenge successfully yielded a high-performing algorithm for a clinical problem.
• The Proteomics Challenge did not produce biologically meaningful results due to issues with data signal and metric selection.
• Underutilized crowdsourcing approaches like gamification and open-source development were identified.

Conclusions:

• Machine learning contests are powerful, open-science tools for addressing complex PD challenges.
• Careful design, including a solvable problem and appropriate scoring metric, is essential for contest success.
• Diverse crowdsourcing techniques hold significant potential to accelerate progress in PD research and clinical care.