Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: May 12, 2026

Examining Gesture Production in the Presence of Communication Challenges
07:18

Examining Gesture Production in the Presence of Communication Challenges

Published on: January 26, 2024

Gestural cue analysis in automated semantic miscommunication annotation.

Masashi Inoue1, Mitsunori Ogihara, Ryoko Hanada

  • 1Collaborative Research Unit, National Institute of Informatics, Tokyo, Japan.

Multimedia Tools and Applications
|April 16, 2013
PubMed
Summary

Detecting semantic miscommunication in videos is hard for machines. This study found that individual gestural cues alone cannot reliably predict miscommunication events in conversations.

Related Concept Videos

Non-Verbal Cues01:29

Non-Verbal Cues

Non-verbal communication extends beyond gestures and facial expressions to include vocal elements known as paralanguage. Paralanguage consists of non-verbal vocal cues such as pitch, loudness, speech rate, pauses, and non-verbal vocalizations like laughter, sighs, and moans. These elements not only accompany speech but also provide critical emotional and contextual information.The Role of Paralanguage in CommunicationParalanguage adds depth to spoken language by conveying emotions and...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Modeling Dyadic Interdependence in Endocrine Functioning: A Multilevel Machine Learning Study of Adults with Cancer and Their Caregivers.

IEEE transactions on bio-medical engineering·2026
Same author

Overlapping clinical features of transfusion-related acute lung injury and anaphylactic shock during general anesthesia: a case report.

JA clinical reports·2026
Same author

Machine Learning Risk Stratification Approach Using Patient-Reported Outcomes for Forecasting Unplanned Health Care Use and Symptom Burden in Cancer Survivors.

JCO clinical cancer informatics·2026
Same author

Impact of pre-compression versus non-compression before parenchyma transection in left-sided pancreatic resection on the rate of clinically relevant pancreatic fistula: multicentre randomized clinical trial.

The British journal of surgery·2025
Same author

Total intravenous anesthesia using midazolam and dexmedetomidine as substitutes for propofol in a pediatric patient with egg allergy and a family history of malignant hyperthermia.

Saudi journal of anaesthesia·2024
Same author

Evaluation of the short-term complications of intracorporeal anastomosis in right-sided colectomy.

Asian journal of endoscopic surgery·2024

Area of Science:

  • Computer Vision
  • Human-Computer Interaction
  • Natural Language Processing

Background:

  • Automated annotation of conversational video is challenging, especially for detecting semantic miscommunication.
  • Machines struggle to identify miscommunications from low-level features, despite their obviousness to humans.
  • Understanding the role of non-verbal cues, particularly gestures, is crucial for improving automated analysis.

Purpose of the Study:

  • To investigate the utility of gestural cues for detecting semantic miscommunication in conversational video.
  • To explore which specific gestural features contribute to or indicate miscommunication.
  • To assess the effectiveness of machine learning classifiers using gestural data.

Main Methods:

  • Extracted nine simple gestural features from gesture data in conversational videos.
Keywords:
Face-to-faceGesturePsychotherapySemantic indexing

Related Experiment Videos

Last Updated: May 12, 2026

Examining Gesture Production in the Presence of Communication Challenges
07:18

Examining Gesture Production in the Presence of Communication Challenges

Published on: January 26, 2024

  • Constructed both simple and complex machine learning classifiers.
  • Evaluated the predictive power of individual gestural features for semantic miscommunication.

    • Main Results:

    • No single gestural feature was found to reliably predict or explain semantic miscommunication.
    • The implicitness and context-dependency of gestures pose significant challenges for automated detection.
    • Gestural cues, while relevant, may not be sufficient on their own for robust miscommunication annotation.

    Conclusions:

    • Individual gestural features are insufficient for accurately predicting semantic miscommunication in automated video analysis.
    • Further research is needed to explore combinations of features or more sophisticated models.
    • The complexity and subtlety of non-verbal communication require advanced methods for computational understanding.