Real-Time Classification of Functional Laryngeal Behaviors From Vocal Fold Kinematics

Develop a real-time prediction model to detect laryngeal behaviors from videolaryngoscopy-based tracking data during clinical examinations. This will improve data collection quality by providing immediate feedback to clinicians and enable future automation pipelines for efficient large-scale analysis.

We trained a stateful residual Gated Recurrent Unit (GRU) model that analyzed the tracking data of 39 laryngeal keypoints, derived from our published keypoint detection model, to predict the patient's laryngeal task state. These included "phonation," "sustained phonation," "swallowing," "idle," "coughing," "sniffing," and "out of view." Model development used 916 state segments comprising 222,065 video frames from 72 laryngoscopy videos. The resulting model was evaluated on an independent dataset comprising 8 videos, 123 segments, and 49,770 frames. Performance was assessed using classification metrics and temporal intersection over union (mIoU).

The model achieved a mean accuracy score of 92% and an mIoU of 0.82 when evaluated on the independent test dataset, indicating agreement with manual annotations and accurate temporal identification of laryngeal task states. At the class level, the model performed consistently across most state categories, with per-class F1-scores on the test set ranging from 83% for phonation to 97% for sniffing. The lowest validation F1 was observed for cough (82% [70%-92%]).

Real-time classification of laryngeal states from videolaryngoscopy-based pose tracking during clinical examinations is feasible. With reliable automated detection of states such as swallowing and phonation, this work establishes a foundation for further automated analysis, interpretation, and documentation of laryngeal pathology.