Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

MultiConsSleepNet: Self-Supervised Contrastive Learning for a Multimodal Consistency-Based Automatic Sleep Staging Model.

Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference·2025
Same author

SemSTNet: Medical EEG Semantic Metric Learning With Class Prototypes Generated by Pretrained Language Model.

IEEE transactions on bio-medical engineering·2025
Same author

SFT-SGAT: A semi-supervised fine-tuning self-supervised graph attention network for emotion recognition and consciousness detection.

Neural networks : the official journal of the International Neural Network Society·2024
Same author

Multimodal Emotion Recognition Based on Facial Expressions, Speech, and EEG.

IEEE open journal of engineering in medicine and biology·2024
Same author

Few-shot EEG sleep staging based on transductive prototype optimization network.

Frontiers in neuroinformatics·2023
Same author

ST-SCGNN: A Spatio-Temporal Self-Constructing Graph Neural Network for Cross-Subject EEG-Based Emotion Recognition and Consciousness Detection.

IEEE journal of biomedical and health informatics·2023
Same journal

Anterior Cingulate Cortex Mediates State-Dependent Prioritization of Distressed Conspecifics.

Brain sciences·2026
Same journal

Hemispherotomy for Pediatric Post-Traumatic Epilepsy.

Brain sciences·2026
Same journal

When Robots Learn: Artificial Intelligence and the Next Human-Centered Era of Neurorehabilitation.

Brain sciences·2026
Same journal

The Association Between Changes in White Matter Microstructure and Cognitive Function in Older Adults with Mild Cognitive Impairment.

Brain sciences·2026
Same journal

Beyond Ventricular Enlargement: Multimodal MRI Assessment Improves Surgical Decision-Making in Normal Pressure Hydrocephalus.

Brain sciences·2026
Same journal

The Effects of Personalized Observation, Execution, and Mental Imagery (POEM) Therapy in Logopenic Primary Progressive Aphasia: A Telepractice-Based Single-Case Study.

Brain sciences·2026
See all related articles

Related Experiment Video

Updated: Dec 7, 2025

Assessment and Communication for People with Disorders of Consciousness
07:37

Assessment and Communication for People with Disorders of Consciousness

Published on: August 1, 2017

9.5K

Advances in Multimodal Emotion Recognition Based on Brain-Computer Interfaces.

Zhipeng He1, Zina Li2, Fuzhou Yang1

  • 1School of Software, South China Normal University, Foshan 528225, China.

Brain Sciences
|October 2, 2020
PubMed
Summary
This summary is machine-generated.

This study reviews multimodal emotion recognition using brain-computer interfaces (BCI). It explores affective BCI (aBCI) systems combining behavior, brain signals, and sensory stimuli for enhanced affective computing.

Keywords:
affective computingbrain–computer interface (BCI)emotion recognitionmultimodal fusion

More Related Videos

Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury
05:51

Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury

Published on: May 15, 2016

9.3K
Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention
06:37

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

Published on: December 15, 2023

4.9K

Related Experiment Videos

Last Updated: Dec 7, 2025

Assessment and Communication for People with Disorders of Consciousness
07:37

Assessment and Communication for People with Disorders of Consciousness

Published on: August 1, 2017

9.5K
Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury
05:51

Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury

Published on: May 15, 2016

9.3K
Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention
06:37

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

Published on: December 15, 2023

4.9K

Area of Science:

  • Affective computing
  • Human-computer interaction
  • Neuroscience

Background:

  • Advancements in portable, noninvasive sensors like brain-computer interfaces (BCI) are driving multimodal emotion recognition.
  • Multimodal approaches integrate diverse data for more robust affective state detection.

Purpose of the Study:

  • To review progress in multimodal emotion recognition specifically within the context of BCI.
  • To categorize and analyze different types of affective BCI (aBCI) systems.

Main Methods:

  • Categorization of aBCI into three types: behavior/brain signal combination, hybrid neurophysiology, and heterogeneous sensory stimuli.
  • Review of representative aBCI systems, detailing design, paradigms, algorithms, and results.

Main Results:

  • Analysis of advantages and limitations for each reviewed aBCI type.
  • Identification of key trends and performance metrics across different multimodal aBCI systems.

Conclusions:

  • Multimodal aBCI shows significant promise for advancing emotion recognition.
  • Future research should address identified challenges and explore new directions in multimodal affective computing.