Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Therapeutic Communication01:30

Therapeutic Communication

6.3K
Communication is a lifelong learning process. Through therapeutic communication, nurses can collect relevant assessment data, provide education and counseling, and interact during nursing interventions. Sending and receiving messages occur through verbal and nonverbal communication techniques and can happen separately or simultaneously.
Verbal communication depends on language or a prescribed way of using words so that people can share information effectively. The critical aspects of verbal...
6.3K
Facial Feedback Hypothesis01:24

Facial Feedback Hypothesis

372
Charles Darwin proposed that facial expressions are an evolutionary adaptation for communication. He argued that these expressions are not influenced by culture but are universal across species. For example, a snarling expression with exposed teeth signals a threat in many animals, including humans. Darwin also suggested that displaying an emotion can intensify the feeling. Smiling, for example, could enhance one's sense of happiness. This idea laid the foundation for understanding the role...
372
Non-Verbal Cues01:29

Non-Verbal Cues

121
Non-verbal communication extends beyond gestures and facial expressions to include vocal elements known as paralanguage. Paralanguage consists of non-verbal vocal cues such as pitch, loudness, speech rate, pauses, and non-verbal vocalizations like laughter, sighs, and moans. These elements not only accompany speech but also provide critical emotional and contextual information.The Role of Paralanguage in CommunicationParalanguage adds depth to spoken language by conveying emotions and...
121
Prosopagnosia01:24

Prosopagnosia

507
Prosopagnosia, also known as face blindness, is the inability to recognize faces. In severe cases, individuals with prosopagnosia may not recognize close family members, including parents and spouses, by their faces. For instance, someone with prosopagnosia might walk past their child in a crowd, only realizing their mistake upon noticing their child's distinctive backpack or favorite jacket. Prosopagnosia specifically impairs facial recognition, while the recognition of other objects or...
507
Nonconscious Mimicry01:13

Nonconscious Mimicry

4.8K
Nonconscious mimicry occurs when individuals alter their mannerisms to match the behaviors and expressions of those nearby, without intention.
4.8K
Channels of Non-Verbal Communication01:28

Channels of Non-Verbal Communication

122
Non-verbal communication plays a critical role in human interaction, influencing how individuals perceive emotions and psychological states. It operates through four primary channels: facial expressions, eye contact, body language, and touch. These non-verbal cues help convey meaning beyond spoken language and are often culturally influenced.Facial Expressions and Emotional RecognitionFacial expressions are among the most powerful and universal forms of non-verbal communication. Research has...
122

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Listening effort measurement by pupillometry under dual-task paradigms: the effect of auditory spectral resolution.

European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery·2026
Same author

HR-ACT (Human-Robot Action) Database: Communicative and noncommunicative action videos featuring a human and a humanoid robot.

Behavior research methods·2026
Same author

A Dual Eye Tracking Study of Joint Attention in Adults with Autism Spectrum Disorder.

Noro psikiyatri arsivi·2025
Same author

Wave 2 of the Multilingual Eye-Movement Corpus (MECO): New text reading data across languages.

Scientific data·2025
Same author

The PSR corpus: A Persian sentence reading corpus of eye movements.

Behavior research methods·2024
Same author

A Naturalistic Setup for Presenting Real People and Live Actions in Experimental Psychology and Cognitive Neuroscience Studies.

Journal of visualized experiments : JoVE·2023

Related Experiment Video

Updated: Nov 12, 2025

Author Spotlight: Deciphering the Cognitive and Neural Mechanisms of Gesture in Communication
07:18

Author Spotlight: Deciphering the Cognitive and Neural Mechanisms of Gesture in Communication

Published on: January 26, 2024

1.1K

Speech Driven Gaze in a Face-to-Face Interaction.

Ülkü Arslan Aydin1, Sinan Kalkan2, Cengiz Acartürk1,3

  • 1Cognitive Science Department, Middle East Technical University, Ankara, Turkey.

Frontiers in Neurorobotics
|March 22, 2021
PubMed
Summary
This summary is machine-generated.

This study explores gaze direction and speech in conversations, finding significant differences based on participant roles. A computational model accurately predicts gaze using speech features, advancing multimodal communication research.

Keywords:
deep learningface-to-face interactiongaze analysismultimodal communicationspeech annotation

More Related Videos

Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
07:36

Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects

Published on: November 30, 2018

16.1K
Group Synchronization During Collaborative Drawing Using Functional Near-Infrared Spectroscopy
07:53

Group Synchronization During Collaborative Drawing Using Functional Near-Infrared Spectroscopy

Published on: August 5, 2022

2.2K

Related Experiment Videos

Last Updated: Nov 12, 2025

Author Spotlight: Deciphering the Cognitive and Neural Mechanisms of Gesture in Communication
07:18

Author Spotlight: Deciphering the Cognitive and Neural Mechanisms of Gesture in Communication

Published on: January 26, 2024

1.1K
Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
07:36

Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects

Published on: November 30, 2018

16.1K
Group Synchronization During Collaborative Drawing Using Functional Near-Infrared Spectroscopy
07:53

Group Synchronization During Collaborative Drawing Using Functional Near-Infrared Spectroscopy

Published on: August 5, 2022

2.2K

Area of Science:

  • Multimodal Communication
  • Human-Computer Interaction
  • Computational Linguistics

Background:

  • Gaze and language are key components of human interaction.
  • Gaze, a non-verbal cue, provides vital social signals but is less studied than language.
  • Understanding gaze-speech dynamics is crucial for developing sophisticated communication models.

Purpose of the Study:

  • Investigate gaze direction (aversion vs. face gaze) and its relationship with speech in face-to-face interactions.
  • Develop and evaluate a computational model for multimodal communication that predicts gaze direction from speech features.
  • Analyze differences in gaze patterns based on participant roles in a controlled setting.

Main Methods:

  • Collected eye-tracking and speech data from 28 pairs of participants in a mock job interview.
  • Annotated speech data using the ISO 24617-2 Standard for Dialogue Act Annotation and social gaze tags.
  • Employed Convolutional Neural Network (CNN) architectures, VGGNet and ResNet, for comparative analysis and gaze prediction.

Main Results:

  • Significant differences in gaze frequency and duration were observed based on participant roles.
  • The ResNet model achieved over 70% accuracy in predicting gaze direction.
  • Established a correlation between specific speech features and gaze behavior.

Conclusions:

  • Gaze patterns are significantly influenced by conversational roles.
  • Computational models, particularly ResNet, show promise in predicting gaze direction from speech.
  • This research contributes to a deeper understanding of multimodal communication and informs the development of AI systems.