Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Language

Language

Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...

Components of Language

Components of Language

Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.

Language Development

Language Development

Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...

Language and Cognition

Language and Cognition

Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.

Physiology of Emotion

Physiology of Emotion

The physiology of emotions is a multifaceted process involving the autonomic nervous system, brain structures, hormones, and neurotransmitters. This intricate interplay dictates how emotions manifest in the body and influence behavior.
Autonomic Nervous System
The autonomic nervous system (ANS) plays a critical role in emotional responses by regulating involuntary physiological functions. It consists of two main components: the sympathetic and parasympathetic systems. The sympathetic system...

Emotional Expression

Emotional Expression

Emotional expression encompasses how individuals convey their emotions through verbal communication and non-verbal cues. These non-verbal actions include facial expressions, body language, and physical gestures, such as frowning or smiling. Among these, facial expressions play a crucial role in emotional expression and are understood universally, indicating a biological basis for how humans communicate emotions.
Universal Facial Expressions
Psychologist Paul Ekman identified seven basic...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Sigma-1 receptor agonist SA4503 improves cerebral protective effects in an extracorporeal cardiopulmonary resuscitation rat model.

JTCVS open·2026

Same author

SARLite: enhanced lightweight YOLO framework for SAR object detection.

Scientific reports·2026

Same author

Outsmarting Metastatic Prostate Cancer: Integration of Imaging, Liquid Biopsies and Biomarkers With Artificial Intelligence.

Technology in cancer research & treatment·2026

Same author

Understanding Personal Protective Equipment Use in Interdisciplinary Medical Settings: Design Explorations for Just-in-Time Compliance Alerts: Improving PPE Practices in Medical Settings Through Alert Design.

DIS. Designing Interactive Systems (Conference)·2026

Same author

STop Clock for Automated Tracking (STAT) during Time-Critical Medical Work: Evaluating the Accuracy and Usability of an AI-Driven Automated Stop Clock.

AMIA ... Annual Symposium proceedings. AMIA Symposium·2026

Same author

SAFE: A Smart Adherence Detection Framework for Monitoring Personal Protective Equipment in Healthcare Settings.

IEEE journal of biomedical and health informatics·2026

Search research articles

Related Experiment Video

Updated: Feb 1, 2026

Experimental Paradigm for Measuring the Effect of Induced Emotion on Grammar Learning

Experimental Paradigm for Measuring the Effect of Induced Emotion on Grammar Learning

Published on: January 29, 2020

DEEP MULTIMODAL LEARNING FOR EMOTION RECOGNITION IN SPOKEN LANGUAGE.

Yue Gu¹, Shuhong Chen¹, Ivan Marsic¹

¹Department of Electrical and Computer Engineering, Rutgers University, Piscataway, NJ, USA.

Proceedings of the ... IEEE International Conference on Acoustics, Speech, and Signal Processing. ICASSP (Conference)

|December 4, 2018

Summary

This study introduces a new deep multimodal framework for predicting human emotions from spoken language. The model effectively combines text and audio features, achieving 60.4% accuracy in emotion recognition.

Keywords:

Emotion recognition deep multimodal learning spoken language

More Related Videos

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language

Published on: October 13, 2018

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties

Published on: September 27, 2020

Related Experiment Videos

Last Updated: Feb 1, 2026

Experimental Paradigm for Measuring the Effect of Induced Emotion on Grammar Learning

Experimental Paradigm for Measuring the Effect of Induced Emotion on Grammar Learning

Published on: January 29, 2020

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language

Published on: October 13, 2018

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties

Published on: September 27, 2020

Area of Science:

Artificial Intelligence
Natural Language Processing
Speech Emotion Recognition

Background:

Accurate human emotion prediction from spoken language is challenging.
Existing methods often struggle to integrate multimodal information effectively.

Purpose of the Study:

To develop a novel deep multimodal framework for sentence-level emotion prediction.
To enhance emotion recognition by leveraging both textual and audio features.

Main Methods:

A hybrid deep multimodal architecture was employed to extract high-level features from text and audio.
Spatial, temporal, and handcrafted features were considered.
A three-layer deep neural network fused features for cross-modal correlation learning.
End-to-end training optimized feature extraction and fusion modules.

Main Results:

The framework achieved 60.4% weighted accuracy in recognizing five emotion categories on the IEMOCAP dataset.
The multimodal approach demonstrated promising performance in speech emotion recognition.

Conclusions:

The proposed deep multimodal framework offers an effective approach for predicting human emotions from spoken language.
Joint training of feature extraction and fusion modules allows for optimal model performance.