Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Attribution Theory00:56

Attribution Theory

13.7K
Behavior is a product of both the situation (e.g., cultural influences, social roles, and the presence of bystanders) and of the person (e.g., personality characteristics). Subfields of psychology tend to focus on one influence or behavior over others. Situationism is the view that our behavior and actions are determined by our immediate environment and surroundings. In contrast, dispositionism holds that our behavior is determined by internal factors (Heider, 1958).
13.7K
Transformers01:26

Transformers

1.7K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.7K
Labeling Emotion01:20

Labeling Emotion

582
Emotional labeling is a cognitive process that involves identifying and naming one's emotions, such as anger, fear, happiness, or sadness. It allows individuals to recognize and express their internal emotional states, a critical aspect of emotional regulation and communication. Labeling emotions requires more than mere recognition; it also involves drawing upon memory and contextual cues to understand the current situation and apply a corresponding emotional label. For instance, feeling...
582
Cognitive Theories: Schachter-Singer Theory of Emotion01:20

Cognitive Theories: Schachter-Singer Theory of Emotion

1.4K
Stanley Schachter and Jerome Singer proposed the two-factor theory of emotion, which emphasizes the interplay between physiological arousal and cognitive labeling in forming emotional experiences. This theory suggests that emotions are not simply a result of physiological responses but rather a combination of these responses and the individual's cognitive interpretation of them.
Physiological Arousal and Cognitive Labeling
According to this theory, when an individual experiences...
1.4K
Cognitive Theories: Lazarus Mediational Theory of Emotion01:17

Cognitive Theories: Lazarus Mediational Theory of Emotion

1.9K
Richard Lazarus' cognitive mediational theory highlights the pivotal role of cognitive appraisal in shaping emotional responses. According to this theory, the evaluation of a stimulus — based on personal values, goals, beliefs, and expectations — mediates the emotional response. This appraisal process is immediate and often occurs unconsciously, influencing the intensity and nature of the resulting emotion.
Cognitive Appraisal and Emotional Response
Lazarus proposed that...
1.9K
Theory of Attribution I: Correspondent Inference Theory01:15

Theory of Attribution I: Correspondent Inference Theory

406
Correspondent inference theory, proposed by Jones and Davis in 1965, seeks to explain how individuals infer stable personality traits from observed behaviors. It suggests that people attribute actions to underlying dispositions rather than external circumstances, particularly when the behavior appears intentional and socially significant.Voluntary Behavior and Dispositional AttributionAccording to this theory, individuals are more likely to attribute behavior to personal traits when it appears...
406

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Explainable ensemble learning framework for predicting industrial and energy sector GHG emissions.

Scientific reports·2026
Same author

Towards decoupling frontend enhancement and backend recognition in monaural robust ASR.

Computer speech & language·2026
Same author

Polyphenols-rich Indian barberry berries extract alleviates inorganic arsenic exposure-induced cognitive impairments and associated gut microflora alterations.

Food research international (Ottawa, Ont.)·2026
Same author

Radiology Challenging Case Competition: A Mentoring/Education Opportunity for Medical Students.

Academic radiology·2026
Same author

Granitic intrusions enhance strain localization and rapid mantle exhumation along an oceanic detachment fault.

Science advances·2026
Same author

CLO26-115: Association Between Geriatric Assessment Variables and Treatment Completion in Older Adults With Triple-Negative Breast Cancer Receiving Neoadjuvant Chemotherapy and Immunotherapy: The MSKCC Experience.

Journal of the National Comprehensive Cancer Network : JNCCN·2026

Related Experiment Video

Updated: May 2, 2026

fMRI Mapping of Brain Activity Associated with the Vocal Production of Consonant and Dissonant Intervals
11:15

fMRI Mapping of Brain Activity Associated with the Vocal Production of Consonant and Dissonant Intervals

Published on: May 23, 2017

7.0K

Bridging Text and Speech for Emotion Understanding: An Explainable Multimodal Transformer Fusion Framework with

Ashutosh Pandey1, Jasmeet Singh1, Maninder Kaur1

  • 1Computer Science Engineering Department, Thapar Institute of Engineering and Technology, Patiala 147001, Punjab, India.

Journal of Intelligence
|December 24, 2025
PubMed
Summary

This study introduces an explainable AI model for emotion recognition, combining language and speech cues for better accuracy. The framework uses multimodal transformers and explainable AI (XAI) to understand affective communication.

Keywords:
emotionexplainable AImultimodal learningspeechtext fusion

More Related Videos

Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique
11:39

Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique

Published on: September 7, 2022

2.1K
Electroencephalographic Signal Acquisition Framework for Neurodiverse: A Case Study of Dolphin-Assisted Therapy
07:21

Electroencephalographic Signal Acquisition Framework for Neurodiverse: A Case Study of Dolphin-Assisted Therapy

Published on: June 27, 2025

631

Related Experiment Videos

Last Updated: May 2, 2026

fMRI Mapping of Brain Activity Associated with the Vocal Production of Consonant and Dissonant Intervals
11:15

fMRI Mapping of Brain Activity Associated with the Vocal Production of Consonant and Dissonant Intervals

Published on: May 23, 2017

7.0K
Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique
11:39

Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique

Published on: September 7, 2022

2.1K
Electroencephalographic Signal Acquisition Framework for Neurodiverse: A Case Study of Dolphin-Assisted Therapy
07:21

Electroencephalographic Signal Acquisition Framework for Neurodiverse: A Case Study of Dolphin-Assisted Therapy

Published on: June 27, 2025

631

Area of Science:

  • Artificial Intelligence
  • Cognitive Science
  • Computational Linguistics

Background:

  • Conversational interactions offer rich linguistic and vocal cues for studying emotion.
  • Existing emotion recognition models often focus on single modalities, limiting understanding of affective communication.

Purpose of the Study:

  • To develop an explainable multimodal transformer framework for advanced emotion understanding.
  • To integrate textual semantics and acoustic prosody for comprehensive affective analysis.

Main Methods:

  • Utilized RoBERTa for textual semantics and WavLM for acoustic prosody.
  • Projected both modalities into a shared latent space for complementary analysis.
  • Embedded explainable AI (XAI) techniques like Integrated Gradients and Occlusion.

Main Results:

  • Achieved 0.83 accuracy across five emotion categories.
  • Demonstrated the model's ability to capture complementary contributions of language and speech.
  • Successfully attributed predictions to specific linguistic tokens and prosodic patterns.

Conclusions:

  • The multimodal AI system enhances emotion recognition accuracy and transparency.
  • Explainable AI techniques align computational mechanisms with human emotion perception.
  • This work supports human-centered emotion recognition through transparent AI.