A Video-Based Measurement Framework for Chewing-Event Detection Using 3D Facial Landmark Dynamics and sEMG-Based...

Accurate measurement of chewing events in natural eating conditions is important for unobtrusive monitoring of feeding behavior and masticatory function. Yet, existing methods often rely on contact sensors, dedicated wearables, or manual annotation. This work presents a non-contact, video-based framework for chewing-event detection using frontal facial video, normalized 3D facial landmark dynamics, and recurrent temporal modeling. To obtain physiologically grounded reference labels, synchronized bilateral anterior temporalis surface electromyography was acquired during real-meal sessions and used to derive chewing-event annotations during dataset construction, whereas inference relied exclusively on video. Facial motion was represented from frame-wise 3D landmarks and processed by recurrent neural networks, with model selection performed through Bayesian hyperparameter optimization. On an independent hold-out test set comprising five sessions and 18,836 frames, the proposed method detected 577 chewing events versus 589 ground truth events, corresponding to a mean absolute error of 4.4 chews/session and a mean absolute percentage error of 4.32%. A comparison with a related rule-based video method from the literature showed substantially larger counting errors (MAE = 39.4, MAPE = 30.39%), particularly in sessions that included concurrent activities such as speaking, suggesting that the proposed approach can reduce counting errors relative to the considered rule-based baseline under the specific meal conditions tested in this feasibility study. The effect of landmark-localization uncertainty on the predicted chewing probability was assessed through Monte Carlo propagation, showing limited impact for most prediction instants and greater sensitivity for intermediate probability values. Finally, the ONNX implementation achieved a mean latency of 8.96 ± 5.74 ms on CPU and 6.89 ± 3.58 ms with CUDA execution on the test workstation, supporting real-time applicability. To support practical deployment, the pipeline was also implemented as a native Kotlin Android application and tested on a commercial tablet, achieving real-time operation at 20 fps.