Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Integrating Multi-Task Eye Tracking and Interpretable Machine Learning for High-Accuracy Screening of Amblyopia in Pediatric Populations.

Journal of eye movement research·2026
Same author

Dual roles of GDNF in enteric glial cell plasticity: direct transdifferentiation via the CaMKII/NeuroD1 pathway and cooperative regulation in a neural stem cell-inducing medium.

Stem cell research & therapy·2026
Same author

Predictive modeling of chronic foot ulcer outcomes using longitudinal photoacoustic imaging.

Npj imaging·2026
Same author

DNER as a novel protein contributes to HSCR pathogenesis: multi-omics combined Mendelian randomization analysis.

Pediatric research·2026
Same author

Neurosense: Bridging Neural Dynamics and Mental Health Through Deep Learning for Brain Health Assessment via Reaction Time and p-Factor Prediction.

Diagnostics (Basel, Switzerland)·2026
Same author

Deep Learning Prediction of Childhood Myopia Progression Using Fundus Image and Refraction Data.

JAMA network open·2026

Related Experiment Video

Updated: Jan 9, 2026

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
05:48

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Published on: August 9, 2024

2.0K

Optimizing Deep Neural Networks for EEG-Based Speech Recognition: A Multimodal Approach to Assistive Communication.

Anarghya Das, Puru Soni, Hubin Zhao

    IEEE Journal of Biomedical and Health Informatics
    |December 8, 2025
    PubMed
    Summary

    This study introduces NeuroSpeech, a multimodal system combining electroencephalography (EEG) and acoustics for improved speech recognition in individuals with impairments. NeuroSpeech demonstrates robust performance, even in noisy conditions, offering a promising assistive technology.

    More Related Videos

    Author Spotlight: Automated Deep Brain Stimulation for Parkinson's Disease - Exploring the Possibilities and Challenges of Home Monitoring
    06:32

    Author Spotlight: Automated Deep Brain Stimulation for Parkinson's Disease - Exploring the Possibilities and Challenges of Home Monitoring

    Published on: July 14, 2023

    1.8K
    Author Spotlight: Advancing Alzheimer's Research – Exploring Early Detection and Multi-Omics Approaches
    09:47

    Author Spotlight: Advancing Alzheimer's Research – Exploring Early Detection and Multi-Omics Approaches

    Published on: December 15, 2023

    1.7K

    Related Experiment Videos

    Last Updated: Jan 9, 2026

    Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
    05:48

    Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

    Published on: August 9, 2024

    2.0K
    Author Spotlight: Automated Deep Brain Stimulation for Parkinson's Disease - Exploring the Possibilities and Challenges of Home Monitoring
    06:32

    Author Spotlight: Automated Deep Brain Stimulation for Parkinson's Disease - Exploring the Possibilities and Challenges of Home Monitoring

    Published on: July 14, 2023

    1.8K
    Author Spotlight: Advancing Alzheimer's Research – Exploring Early Detection and Multi-Omics Approaches
    09:47

    Author Spotlight: Advancing Alzheimer's Research – Exploring Early Detection and Multi-Omics Approaches

    Published on: December 15, 2023

    1.7K

    Area of Science:

    • Neuroscience
    • Artificial Intelligence
    • Assistive Technology

    Background:

    • Speech recognition for individuals with impairments is challenging due to atypical speech patterns.
    • Traditional acoustic-only models struggle with these variations.
    • There is a need for robust and efficient speech recognition solutions for impaired communication.

    Purpose of the Study:

    • To introduce NeuroSpeech, a novel multimodal framework integrating electroencephalography (EEG) and acoustic features.
    • To enhance speech recognition accuracy, robustness, and efficiency for individuals with speech impairments.
    • To investigate the role of EEG as a noise-robust complementary signal.

    Main Methods:

    • Developed NeuroSpeech, a multimodal framework combining EEG and acoustic features.
    • Utilized a large-scale random search to identify optimal EEG encoder configurations and feature extraction parameters.
    • Employed Explainable AI (XAI) methods, specifically SHAP, for model interpretability.
    • Evaluated performance on Spanish (UNLP-CONICET) and English (KaraOne) datasets under clean and noisy conditions.

    Main Results:

    • NeuroSpeech achieved near-perfect accuracy in clean conditions (F1=0.986 Spanish; 0.837 English).
    • Maintained strong performance in noisy conditions (F1=0.92 Spanish; 0.70 English), outperforming traditional models like Whisper.
    • Demonstrated EEG's effectiveness as a noise-robust complementary signal.
    • Showcased NeuroSpeech's lightweight nature (1-30M parameters) and near-real-time inference (10-18ms/sample).

    Conclusions:

    • NeuroSpeech significantly improves speech recognition for individuals with impairments by integrating neural and acoustic data.
    • The framework offers enhanced robustness in noisy environments, a critical factor for real-world applications.
    • NeuroSpeech represents a promising advancement in assistive technologies, enabling better communication for those with speech disorders.