Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Hearing a face: cross-modal speaker matching using isolated visible speech.

Lawrence D Rosenblum1, Nicolas M Smith, Sarah M Nichols

  • 1Department of Psychology, University of California, Riverside, CA 92521, USA. rosenblu@citrus.ucr.edu

Perception & Psychophysics
|April 19, 2006
PubMed
Summary

This study shows that visual speech movements alone can help identify a speaker. Maintaining natural speech dynamics in point-light displays improved cross-modal speaker matching accuracy.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Autologous stem cell transplantation for secondary central nervous system lymphoma: a multicentre retrospective analysis.

BMC cancer·2026
Same author

Transient vision loss from septic emboli mimicking giant cell arteritis.

BMJ case reports·2026
Same author

Assessing the therapeutic effects of DPP4 Inhibitors on TGF-β2-induced Lens Opacity.

Experimental eye research·2026
Same author

A global scoping review to inform the recruitment and retention of Aboriginal and Torres Strait Islander nursing and midwifery academics.

Nurse education in practice·2025
Same author

Upfront memory T cell add-back with haploidentical TCRαβ-depleted graft in adults with haematological malignancies: a nationwide, multicentre, single-arm, prospective study.

Bone marrow transplantation·2025
Same author

Assessing the digital health maturity of general practice in Australia: results from a cross-sectional national survey.

Australian journal of primary health·2025

Area of Science:

  • Auditory and Visual Perception
  • Speech Processing
  • Human-Computer Interaction

Background:

  • Cross-modal perception links different sensory inputs.
  • Speaker recognition often relies on both auditory and visual cues.
  • The role of visual speech dynamics in speaker identification is not fully understood.

Purpose of the Study:

  • To investigate if visible speech movements, isolated via point-light technique, can support cross-modal speaker matching.
  • To determine the importance of natural speech dynamics for accurate speaker identification from visual cues.

Main Methods:

  • Subjects matched voices to speaking point-light faces based on speaker identity.
  • Five experimental conditions varied the presentation of speech dynamics (maintained vs. distorted/deleted).

Related Experiment Videos

  • Some conditions controlled for video frame content across dynamic variations.

    • Main Results:

    • Matching performance was significantly better when natural speech dynamics were preserved.
    • Distorting or deleting speech dynamics impaired the ability to match speakers.
    • Results held even when video frames were equated between conditions.

    Conclusions:

    • Visible speech movements contain sufficient information for cross-modal speaker identification.
    • The idiosyncratic dynamics of speech movements are crucial for accurate visual speaker recognition.
    • This research highlights the importance of dynamic visual cues in multimodal communication.