Jove
Visualize
Contact Us

Related Experiment Videos

Text generation from Taiwanese Sign Language using a PST-based language model for augmentative communication.

Chung-Hsien Wu1, Yu-Hsien Chiu, Chi-Shiang Guo

  • 1Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan, ROC. chwu@csie.ncku.edu.tw

IEEE Transactions on Neural Systems and Rehabilitation Engineering : a Publication of the IEEE Engineering in Medicine and Biology Society
|December 24, 2004
PubMed
Summary

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

GLR channels are required for rhythmic scent emission in Phalaenopsis bellina.

The Plant journal : for cell and molecular biology·2026
Same author

MoodSensing: A smartphone app for digital phenotyping and assessment of bipolar disorder.

Psychiatry research·2024
Same author

Systematic designs of single metal compounds synthesized using ammonia fluoride-based complex as structure directing agents for energy storage.

Journal of colloid and interface science·2023
Same author

Ligand Incorporating Sequence-dependent ZIF67 Derivatives as Active Material of Supercapacitor: Competition between Ammonia Fluoride and 2-Methylimidazole.

ACS applied materials & interfaces·2022
Same author

Long-term musical training induces white matter plasticity in emotion and language networks.

Human brain mapping·2022
Same author

PADAr: physician-oriented artificial intelligence-facilitating diagnosis aid for retinal diseases.

Journal of medical imaging (Bellingham, Wash.)·2022
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

This study introduces a novel method for converting Taiwanese Sign Language (TSL) into Chinese sentences using a predictive sentence template (PST) language model. This approach significantly enhances communication for individuals with hearing impairments.

Area of Science:

  • Natural Language Processing
  • Human-Computer Interaction
  • Assistive Technology

Background:

  • Effective communication for individuals with hearing impairments is crucial.
  • Existing methods for converting sign language to text often struggle with ill-formed input.
  • Bridging the gap between signed and written language requires sophisticated language modeling.

Purpose of the Study:

  • To develop a novel approach for generating grammatical Chinese sentences from ill-formed Taiwanese Sign Language (TSL).
  • To improve the efficiency and accuracy of text generation for assistive communication technologies.
  • To evaluate the effectiveness of the proposed system as a practical communication aid for deaf students.

Main Methods:

  • A sign icon-based virtual keyboard was developed for accessing a sign database.

Related Experiment Videos

  • A predictive sentence template (PST) language model (LM) integrating n-gram LM and linguistic constraints was proposed.
  • Phrase formation rules based on trigger pair category were derived for sentence pattern expansion.
  • The PST tree was trained on a corpus from deaf schools to model signed-written Chinese correspondence.

    • Main Results:

    • The proposed methods improved text generation efficiency and word prediction accuracy.
    • The input rate for text generation was significantly enhanced.
    • A reading-comprehension training program with deaf students showed marked improvements in literacy aptitude.
    • Subjective satisfaction levels among participants were significantly higher.

    Conclusions:

    • The novel PST language model effectively translates ill-formed TSL into grammatical Chinese sentences.
    • The developed system serves as a practical and effective communication aid for people with hearing impairments.
    • The approach demonstrates potential for advancing assistive technologies in sign language translation and literacy development.