Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Facial Feedback Hypothesis01:24

Facial Feedback Hypothesis

139
Charles Darwin proposed that facial expressions are an evolutionary adaptation for communication. He argued that these expressions are not influenced by culture but are universal across species. For example, a snarling expression with exposed teeth signals a threat in many animals, including humans. Darwin also suggested that displaying an emotion can intensify the feeling. Smiling, for example, could enhance one's sense of happiness. This idea laid the foundation for understanding the role...
139

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Adaptive Gating and Focal Debiasing for robust few-shot retinal disease classification.

Frontiers in artificial intelligence·2026
Same author

A systematic review investigating emerging trends between Extreme Weather Events (EWEs) and infectious disease outbreaks in South Africa.

Frontiers in public health·2026
Same author

Editorial: Soft computing and machine learning applications for healthcare systems.

Frontiers in artificial intelligence·2026
Same author

Correction: Zeng et al. High-Performance Silicon Nanowire Array Biosensor for Combined Detection of Colorectal Cancer Biomarkers. <i>Micromachines</i> 2025, <i>16</i>, 1089.

Micromachines·2025
Same author

High-Performance Silicon Nanowire Array Biosensor for Combined Detection of Colorectal Cancer Biomarkers.

Micromachines·2025
Same author

Deep Learning Techniques for Prostate Cancer Analysis and Detection: Survey of the State of the Art.

Journal of imaging·2025

Related Experiment Video

Updated: Jun 24, 2025

Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury
05:51

Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury

Published on: May 15, 2016

9.0K

An enhanced speech emotion recognition using vision transformer.

Samson Akinpelu1, Serestina Viriri2, Adekanmi Adegun1

  • 1School of Mathematics, Statistics and Computer Science, University of KwaZulu-Natal, Durban, 4001, South Africa.

Scientific Reports
|June 7, 2024
PubMed
Summary

This study introduces a novel method using a Vision Transformer (ViT) for speech emotion recognition (SER), achieving high accuracy. The approach effectively captures emotional cues from mel spectrograms, enhancing human-computer interaction systems.

Keywords:
CNNDeep learningHuman–computer interactionMel spectrogramSpeech emotion recognitionVision transformer

More Related Videos

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention
06:37

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

Published on: December 15, 2023

2.7K
A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images
04:23

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images

Published on: April 21, 2023

1.8K

Related Experiment Videos

Last Updated: Jun 24, 2025

Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury
05:51

Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury

Published on: May 15, 2016

9.0K
Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention
06:37

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

Published on: December 15, 2023

2.7K
A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images
04:23

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images

Published on: April 21, 2023

1.8K

Area of Science:

  • Computer Science
  • Artificial Intelligence
  • Human-Computer Interaction

Background:

  • Speech emotion recognition (SER) traditionally relies on acoustic features.
  • Deep learning and computer vision advancements enable multimodal SER.
  • Integrating visual features can significantly improve SER performance.

Purpose of the Study:

  • To propose a novel method for enhancing speech emotion recognition (SER) using a lightweight Vision Transformer (ViT).
  • To leverage ViT's capability in capturing spatial and high-level features from mel spectrograms for emotion detection.
  • To evaluate the proposed method's effectiveness on benchmark datasets.

Main Methods:

  • Utilized a lightweight Vision Transformer (ViT) model.
  • Processed mel spectrograms as input to the ViT model.
  • Employed a non-overlapping patch-based feature extraction method.

Main Results:

  • Achieved high accuracy rates: 98% on TESS, 91% on EMODB, and 93% on TESS-EMODB.
  • Demonstrated considerable improvement in SER accuracy and generalizability.
  • The non-overlapping patch-based feature extraction method significantly enhanced SER.

Conclusions:

  • The proposed ViT-based method offers a significant advancement in speech emotion recognition.
  • Vision Transformer models show great potential for integration into SER systems.
  • This research opens new avenues for real-world applications requiring accurate emotion recognition from speech.