Related Experiment Video
Updated: May 26, 2025
12:43
A Protocol for Comprehensive Assessment of Bulbar Dysfunction in Amyotrophic Lateral Sclerosis ALS
Published on: February 21, 2011
34.4K
3D markerless tracking of speech movements with submillimeter accuracy
Austin Lovell1, James Liu2, Arielle Borovsky3
1Department of Computer Science, Purdue University, West Lafayette, IN, United States.
Biorxiv : the Preprint Server for Biology
|February 24, 2025
Summary
A new markerless system precisely tracks 3D speech movements in adults and children using integrated AI models. This technology offers submillimeter accuracy without physical sensors, advancing speech research.
Area of Science:
- Computational linguistics
- Biomedical engineering
- Machine learning applications in speech science
Background:
- Accurate measurement of complex speech movements is crucial for intelligible communication.
- Traditional sensor-based methods for tracking oral articulators are intrusive and challenging for certain populations, like young children.
- Markerless facial landmark tracking shows promise but lacks validated 3D precision and accuracy, especially for pediatric speech.
Purpose of the Study:
- To develop and validate a novel, markerless 3D speech movement tracking approach.
- To assess the precision and accuracy of the integrated system, comparing it to existing methods.
- To determine the system's efficacy in tracking speech movements in both adults and young children.
Main Methods:
- Developed a novel framework integrating Shape Preserving Facial Landmarks with Graph Attention Networks (SPIGA) for landmark detection and CoTracker, a transformer-based model, for dense point tracking.
- Validated the approach by assessing 3D tracking precision (standard deviation) and accuracy (Root Mean Square Error against electromagnetic articulography).
- Evaluated performance across adult participants and young children (3- and 4-year-olds).
Main Results:
- The integrated SPIGA and CoTracker approach achieved higher precision (≈ 0.15 mm SD) compared to SPIGA alone (≈ 0.35 mm SD).
- 3D tracking accuracy was comparable to electromagnetic articulography (≈ 0.29 mm RMSE).
- The system demonstrated similar performance in tracking speech movements in both adults and young children.
Conclusions:
- The novel markerless framework provides submillimeter precision and accuracy for 3D speech movement tracking, suitable for diverse age groups.
- This open-source, integrated approach enhances accessibility and reduces computational costs for speech analysis.
- The study serves as a proof-of-concept for improving markerless tracking applications in biology and neuroscience through model integration.
