Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Does Gamma Knife Stereotactic Radiosurgery Preserve Hearing on Short- and Long-Term Follow-Up?

The Laryngoscope·2026

Same author

Otological conditions and autism spectrum disorder in Medicaid beneficiaries aged 1 to 20 years: a cross-sectional study.

BMC pediatrics·2026

Same author

Multi-Task Assessment of Context-Specific Gait Changes to Virtual Reality-Based Visual Perturbations.

The Laryngoscope·2026

Same author

Reply: comment on "Superior semicircular abnormalities are more common in patients presenting to the emergency room with unexplained vertigo".

Clinical imaging·2026

Same author

Music Enjoyment and Emotional Response to Music in Cochlear Implant Users are Independent of Listening Modality.

Otology & neurotology : official publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology·2025

Same author

Cochlear Implants Impair Emotional Responses to Music in Individuals With Single-Sided Deafness.

Otology & neurotology : official publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology·2025

Search research articles

Related Experiment Video

Updated: Mar 20, 2026

Author Spotlight: An Automated Method for Assessing Visual Acuity in Infants and Toddlers Using an Eye-Tracking System

Author Spotlight: An Automated Method for Assessing Visual Acuity in Infants and Toddlers Using an Eye-Tracking System

Published on: March 17, 2023

Development of a Low-Cost, Noninvasive, Portable Visual Speech Recognition Program.

Gavriel D Kohlberg¹, Ya'akov Kobi Gal², Anil K Lalwani³

¹Department of Otolaryngology-Head and Neck Surgery, Columbia University College of Physicians and Surgeons, New York, New York, USA.

The Annals of Otology, Rhinology, and Laryngology

|May 22, 2016

Summary

This study developed a visual speech recognition program (VSRP) using Kinect to convert lip movements into speech. The VSRP achieved 77.2% accuracy, offering a new communication method for patients unable to speak.

Keywords:

communication aids for disabled lipreading silent speech interface user computer interface visual speech recognition

More Related Videos

Development of an Audio-based Virtual Gaming Environment to Assist with Navigation Skills in the Blind

Development of an Audio-based Virtual Gaming Environment to Assist with Navigation Skills in the Blind

Published on: March 27, 2013

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Published on: August 9, 2024

Related Experiment Videos

Last Updated: Mar 20, 2026

Author Spotlight: An Automated Method for Assessing Visual Acuity in Infants and Toddlers Using an Eye-Tracking System

Author Spotlight: An Automated Method for Assessing Visual Acuity in Infants and Toddlers Using an Eye-Tracking System

Published on: March 17, 2023

Development of an Audio-based Virtual Gaming Environment to Assist with Navigation Skills in the Blind

Development of an Audio-based Virtual Gaming Environment to Assist with Navigation Skills in the Blind

Published on: March 27, 2013

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Published on: August 9, 2024

Area of Science:

Biomedical Engineering
Speech Technology
Human-Computer Interaction

Background:

Aphonia resulting from tracheostomy and laryngectomy severely impairs communication.
Existing communication methods for aphonic patients rely on limited gestures and facial expressions.

Purpose of the Study:

To develop a low-cost, noninvasive, portable visual speech recognition program (VSRP).
To convert articulatory facial movements into audible speech for aphonic patients.

Main Methods:

A Microsoft Kinect-based VSRP was developed to capture lip movement coordinates.
An artificial neural network classifier was trained using articulatory speech movements for 12 sentences.
Classifier accuracy was evaluated on unseen articulatory speech data.

Main Results:

The VSRP achieved a 77.2% accuracy rate (65.0%-87.6% across 5 subjects) on a 12-sentence dataset.
The system demonstrated rapid classification with a mean time of 2.03 milliseconds per sentence.
Successful implementation and testing were conducted on 5 human subjects.

Conclusions:

A feasible, low-cost, noninvasive VSRP using Kinect technology was demonstrated.
The VSRP accurately predicts speech from articulation movements in clinically relevant timeframes.
This VSRP holds potential as a novel communication device for patients with speech loss.