Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Maximizing sinusoidal channels of HZSM-5 for high shape-selectivity to p-xylene.

Nature communications·2019
Same author

Correction: Loss of the novel mitochondrial protein FAM210B promotes metastasis via PDK4-dependent metabolic reprogramming.

Cell death & disease·2019
Same author

RAD54B potentiates tumor growth and predicts poor prognosis of patients with luminal A breast cancer.

Biomedicine & pharmacotherapy = Biomedecine & pharmacotherapie·2019
Same author

Clinical management of malignant ovarian germ cell tumors: A 26-year experience in a tertiary care institution.

Surgical oncology·2019
Same author

Construction of a sp<sup>3</sup> /sp<sup>2</sup> Carbon Interface in 3D N-Doped Nanocarbons for the Oxygen Reduction Reaction.

Angewandte Chemie (International ed. in English)·2019
Same author

A versatile route to fabricate single atom catalysts with high chemoselectivity and regioselectivity in hydrogenation.

Nature communications·2019

Related Experiment Video

Updated: May 24, 2025

Ultrasound Images of the Tongue: A Tutorial for Assessment and Remediation of Speech Sound Errors
08:32

Ultrasound Images of the Tongue: A Tutorial for Assessment and Remediation of Speech Sound Errors

Published on: January 3, 2017

21.6K

End-to-End Mandarin Speech Reconstruction Based on Ultrasound Tongue Images Using Deep Learning.

Fengji Li, Fei Shen, Ding Ma

    IEEE Transactions on Neural Systems and Rehabilitation Engineering : a Publication of the IEEE Engineering in Medicine and Biology Society
    |March 3, 2025
    PubMed
    Summary

    This study reconstructs speech for laryngectomees using ultrasound tongue images and deep learning. The method shows promising results for restoring intelligible Mandarin speech, aiding speech rehabilitation.

    More Related Videos

    Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
    05:56

    Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

    Published on: April 14, 2023

    2.3K
    Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
    05:48

    Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

    Published on: August 9, 2024

    1.4K

    Related Experiment Videos

    Last Updated: May 24, 2025

    Ultrasound Images of the Tongue: A Tutorial for Assessment and Remediation of Speech Sound Errors
    08:32

    Ultrasound Images of the Tongue: A Tutorial for Assessment and Remediation of Speech Sound Errors

    Published on: January 3, 2017

    21.6K
    Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
    05:56

    Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

    Published on: April 14, 2023

    2.3K
    Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
    05:48

    Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

    Published on: August 9, 2024

    1.4K

    Area of Science:

    • Biomedical Engineering
    • Speech Science
    • Artificial Intelligence

    Background:

    • Laryngectomy often results in speech loss, causing significant distress.
    • Intact articulatory organs in laryngectomees highlight the potential for speech rehabilitation.
    • Utilizing articulatory motion information is key to restoring speech function.

    Purpose of the Study:

    • To develop a deep learning-based method for speech reconstruction from ultrasound tongue images.
    • To restore intelligible speech for individuals who have undergone laryngectomy.
    • To leverage articulatory motion data for speech rehabilitation.

    Main Methods:

    • Collected simultaneous ultrasound tongue images and speech data using a Mandarin corpus.
    • Developed an end-to-end speech reconstruction model using adversarial neural networks.
    • Incorporated a pretrained feature extractor, upsampling block, and discriminators for speech generation and fidelity.

    Main Results:

    • Reconstructed speech demonstrated high intelligibility for Mandarin phonemes and tones.
    • Achieved a phoneme character error rate of 0.2544 and a tone error rate of 0.1784.
    • Objective and subjective evaluations confirmed high similarity and acceptable naturalness of the reconstructed speech.

    Conclusions:

    • The proposed deep learning method can reconstruct tonal Mandarin speech from ultrasound tongue images.
    • The findings support the potential for speech rehabilitation in laryngectomees.
    • Future work should focus on optimizing the model for specific laryngectomee conditions and expanding datasets.