Massive online data annotation, crowdsourcing to generate high quality sleep spindle annotations from EEG data.
Karine Lacourse1, Ben Yetton2, Sara Mednick2
1Centre d'études avancées en médecine du sommeil, Montréal, Canada. karinelack@gmail.com.
Scientific Data
|June 21, 2020
Summary
Automated sleep spindle detection requires large, validated datasets. Crowdsourcing via the MODA platform created an open-source dataset, enabling robust training and validation of automated sleep spindle detection algorithms.
Area of Science:
- Neuroscience
- Sleep Science
- Computational Biology
Background:
- Spindle event detection is crucial for sleep analysis.
- Manual detection by experts is time-consuming and expensive.
- Automated algorithms need robust datasets for training and validation.
Purpose of the Study:
- To create a large, open-source dataset of human-scored sleep spindles using crowdsourcing.
- To evaluate the performance of human scorers and existing automated spindle detection algorithms.
- To demonstrate the utility of the MODA platform for generating standardized biological signal datasets.
Main Methods:
- Utilized the Massive Online Data Annotation (MODA) platform for crowdsourced data collection.
- Collected 5342 human-scored sleep spindles from 180 subjects.
- Evaluated three human scorer subtypes (experts, researchers, non-experts) and seven automated algorithms.
Main Results:
- Crowdsourcing generated a high-quality, open-source dataset for sleep spindle research.
- Only two of seven tested algorithms achieved performance comparable to human experts.
- Significant age and sex differences were observed in spindle characteristics.
Conclusions:
- The MODA platform effectively generates valid, standardized datasets for biological signal research.
- The developed dataset facilitates training, validation, and comparison of automated sleep spindle detectors.
- Findings highlight the need for robust algorithms and underscore demographic variations in sleep spindles.
