Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Automatic lipreading

P L Silsbee1, A C Bovik

  • 1Dept. of Elect. and Computer Engineering, University of Texas, Austin 78712-1084.

Biomedical Sciences Instrumentation
|January 1, 1993
PubMed
Summary
This summary is machine-generated.

An automatic visual lipreading system enhances automatic speech recognition (ASR) accuracy. Combining audio and visual speech recognition significantly reduces errors, especially in noisy conditions.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Automatic segmentation of inorganic nanoparticles in BF TEM micrographs.

Ultramicroscopy·2018
Same author

The effect of median filtering on edge estimation and detection.

IEEE transactions on pattern analysis and machine intelligence·2011
Same author

Eye movements selective for spatial frequency and orientation during active visual search.

Vision research·2008
Same author

GAFFE: a gaze-attentive fixation finding engine.

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society·2008
Same author

Nonlinear image estimation using piecewise and local image models.

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society·2008
Same author

Multiresolution 3-D range segmentation using focus cues.

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society·2008
Same journal

EFFECT OF FILTERING KINEMATICS ON FINITE ELEMENT SIMULATIONS OF HEAD IMPACTS IN HIGH SCHOOL FEMALE LACROSSE.

Biomedical sciences instrumentation·2026
Same journal

INHIBITING THE INHIBITOR: WOULD TARGETING PAI-1 RESULT IN A LOW-DOSE, WELL-TOLERATED TREATMENT OF EMPYEMA?

Biomedical sciences instrumentation·2026
Same journal

QUANTIFYING HEAD IMPACT EXPOSURE, MECHANISMS AND KINEMATICS USING INSTRUMENTED MOUTHGUARDS IN MALE HIGH SCHOOL LACROSSE.

Biomedical sciences instrumentation·2026
Same journal

UTILITY OF CHEST ULTRASONOGRAPHY IN QUANTIFYING CLOT AND PLEURAL EFFUSION VOLUME IN PRECLINICAL MODELS OF PLEURAL DISEASE.

Biomedical sciences instrumentation·2026
Same journal

THE ROLE OF PERIPHERAL VISION IN ENHANCING BALANCE AND POSTURAL STABILITY: INSIGHTS FROM CENTRAL VISION OBSTRUCTION.

Biomedical sciences instrumentation·2025
Same journal

IMPACT OF NON-SKID SOCKS AND ANTERIOR WEIGHT ON POSTURAL RESPONSE AND STABILITY DURING PERTURBATION.

Biomedical sciences instrumentation·2025
See all related articles

Area of Science:

  • Computer Science
  • Artificial Intelligence
  • Signal Processing

Background:

  • Automatic speech recognition (ASR) systems often struggle with noisy environments.
  • Visual cues from lip movements can provide complementary information to audio signals.
  • Integrating multimodal information can improve the robustness of speech recognition.

Purpose of the Study:

  • To develop and evaluate an automatic visual lipreading system.
  • To assess the performance of the visual system alone and in conjunction with an audio-based ASR.
  • To determine the impact of the combined system on speech recognition accuracy, particularly in the presence of white noise.

Main Methods:

  • Extraction of geometrical and local brightness parameters from video image sequences.

Related Experiment Videos

  • Application of standard automatic speech recognition techniques to visual data.
  • Testing performance on discriminating 22 consonants with and without added white noise.
  • Comparison of unimodal audio, unimodal visual, and bimodal audio-visual system performance.

    • Main Results:

    • The visual lipreading system achieved approximately 47% accuracy on its own.
    • The combined audio-visual system significantly outperformed either individual system.
    • Error reduction in the combined system reached up to 61% in noisy conditions.
    • The benefit of the combined system was more pronounced with increasing levels of white noise.

    Conclusions:

    • Automatic visual lipreading is a viable supplement to standard automatic speech recognition.
    • Multimodal speech recognition integrating audio and visual data offers enhanced robustness and accuracy.
    • The developed system shows particular promise for improving speech recognition in challenging acoustic environments.