Continuous, Contactless, and Multimodal Pain Assessment During Surgical Interventions

Goal: We introduce a continuous, multimodal pain classification technique that utilizes camera-based data conducted in clinical settings. Methods: We integrate facial Action Units (AUs) obtained from samples with sequential vital parameters extracted from video data, and systematically validate the practicality of measuring heart rate variability (HRV) from video-derived photoplethysmographic signals against traditional sensor-based electrocardiogram measurements. Video-based AUs and HRV metrics acquired from ultra-short-term processing are combined into an automated, contactless, multimodal algorithm for binary pain classification. Utilizing logistic regression alongside leave-one-out cross-validation, this approach is developed and validated using the BioVid Heat Pain Database and subsequently tested with our surgical Individual Patient Data. Results: We achieve an F1-score of 53% on the BioVid Heat Pain Database and 48% on our Individual Patient Data with ultra-short-term processing. Conclusion: Our approach provides a robust foundation for future multimodal pain classification utilizing vital signs and mimic parameters from 5.5 s camera recordings.