Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same authorSame Topic

Large language model derived regular expressions for sleep phenotyping from electronic health record: a feasibility study.

Sleep advances : a journal of the Sleep Research Society·2026
Same author

Invited Commentary: Sharper than a Scalpel: How Imaging Redefines What It Means to Operate on the Brain.

Radiographics : a review publication of the Radiological Society of North America, Inc·2026
Same author

Gastrointestinal Dysfunction in Critically Ill Patients With Traumatic Brain Injury: Clinical Implications and Putative Mechanisms: a Narrative Review.

Current neurology and neuroscience reports·2026
Same author

Automated Prediction of Glasgow Coma Scale Scores From Unstructured Electronic Health Records Using Natural Language Processing: Development and Validation Study.

Journal of medical Internet research·2026
Same author

Neuroprotective Treatment of Postanoxic Encephalopathy: Review of the Literature and Ongoing Trials.

Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society·2026
Same author

Cost-Effectiveness of EEG Monitoring in Hypoxic-Ischemic Brain Injury After Cardiac Arrest.

Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society·2026

Related Experiment Video

Updated: Sep 12, 2025

Author Spotlight: Advancing the Study of Brain-Heart Interplay with a Comprehensive EEGLAB Plugin for Multimodal Signal Analysis
08:22

Author Spotlight: Advancing the Study of Brain-Heart Interplay with a Comprehensive EEGLAB Plugin for Multimodal Signal Analysis

Published on: April 26, 2024

2.2K

Evaluating crowdsourcing for ICU EEG annotation: A comparison with expert performance.

Wan-Yee Kong1,2, Fábio A Nascimento3, Aaron Struck4

  • 1Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA.

Epilepsia
|August 6, 2025
PubMed
Summary

Crowdsourcing EEG annotations using a mobile app showed that weighted majority votes from non-experts were comparable to expert performance in identifying seizures and rhythmic patterns. This approach could accelerate the creation of large datasets for automated detection algorithms.

Keywords:
EEGannotationcrowdsourcingmachine learningseizures rhythmic and periodic patterns

More Related Videos

Assessment and Communication for People with Disorders of Consciousness
07:37

Assessment and Communication for People with Disorders of Consciousness

Published on: August 1, 2017

9.2K
Lesion Explorer: A Video-guided, Standardized Protocol for Accurate and Reliable MRI-derived Volumetrics in Alzheimer's Disease and Normal Elderly
12:50

Lesion Explorer: A Video-guided, Standardized Protocol for Accurate and Reliable MRI-derived Volumetrics in Alzheimer's Disease and Normal Elderly

Published on: April 14, 2014

40.3K

Related Experiment Videos

Last Updated: Sep 12, 2025

Author Spotlight: Advancing the Study of Brain-Heart Interplay with a Comprehensive EEGLAB Plugin for Multimodal Signal Analysis
08:22

Author Spotlight: Advancing the Study of Brain-Heart Interplay with a Comprehensive EEGLAB Plugin for Multimodal Signal Analysis

Published on: April 26, 2024

2.2K
Assessment and Communication for People with Disorders of Consciousness
07:37

Assessment and Communication for People with Disorders of Consciousness

Published on: August 1, 2017

9.2K
Lesion Explorer: A Video-guided, Standardized Protocol for Accurate and Reliable MRI-derived Volumetrics in Alzheimer's Disease and Normal Elderly
12:50

Lesion Explorer: A Video-guided, Standardized Protocol for Accurate and Reliable MRI-derived Volumetrics in Alzheimer's Disease and Normal Elderly

Published on: April 14, 2014

40.3K

Area of Science:

  • Neuroscience
  • Medical Informatics
  • Computational Biology

Background:

  • Accurate detection of seizures and rhythmic or periodic patterns (SRPPs) on electroencephalography (EEG) is vital for managing critically ill neurological patients.
  • Automated EEG analysis methods require large, expert-annotated datasets, but neurophysiologist availability limits expert annotation.
  • Crowdsourcing offers a potential solution to scale up EEG data annotation.

Purpose of the Study:

  • To evaluate the feasibility of using crowdsourcing for annotating EEG recordings.
  • To compare the performance of non-expert crowdsourced annotations against expert neurophysiologists in identifying six types of SRPPs.

Main Methods:

  • An EEG scoring contest was conducted via a mobile app, engaging 1542 participants (8 experts, 1534 non-experts).
  • Participants annotated 478,834 short EEG epochs across six SRPPs: seizures, generalized and lateralized periodic discharges (GPDs, LPDs), and generalized and lateralized rhythmic delta activity (GRDA, LRDA), plus 'Other'.
  • Performance was assessed using pairwise agreement, Fleiss' kappa for experts, and accuracy comparisons between experts and the crowd via individual and weighted majority votes.

Main Results:

  • The crowd's individual, non-weighted votes were inferior to experts for overall and specific SRPP identification.
  • Using weighted majority votes, the crowd achieved non-inferior overall SRPP identification accuracy (.70, 95% CI: .69-.70) compared to experts (.68, 95% CI: .68-.70).
  • The crowd matched or exceeded expert performance for most SRPPs, excluding LPDs and 'Other'; no single expert outperformed the crowd overall.

Conclusions:

  • Crowd reviewers show promise for achieving expert-level EEG annotations, potentially enabling the development of larger, more diverse datasets for automated detection algorithms.
  • This proof-of-concept study suggests crowdsourcing is a viable method for EEG annotation.
  • Further research is needed to address challenges like participant calibration and the absence of gold-standard labels in real-world applications.