Cross-spectral fusion of thermal and RGB imaging for objective pain estimation

Pain assessment remains challenging for patients unable to verbally communicate, including neonates, cognitively impaired individuals, sedated patients, and those who suppress expressions due to cultural norms or stoicism. We demonstrate that integrating thermal imaging with RGB facial expression analysis provides more accurate and robust pain intensity estimation than either modality alone. Our dual-camera system records synchronized thermal and RGB video, processed through a cross-spectral attention fusion (CSAF) model with a temporal transformer for continuous 0-10 scale pain prediction. In a controlled laboratory pain induction study, 50 healthy adults (ages 21-68, 87.3 h video) underwent Cold Pressor Test and pressure algometry protocols; our system achieves MAE  =  0.79, representing a 33.1% improvement over RGB-only (MAE  =  1.18, p < 0.001). In a real-world clinical postoperative monitoring study, 30 surgical patients (ages 31-74, 17.7 h video) recovering from abdominal surgery were monitored; our system achieves MAE  =  1.08, representing a 28.5% improvement over RGB-only (MAE  =  1.51, p < 0.001). Across the combined cohort (n = 80), MAE  =  0.87 (29.3% overall improvement over RGB-only). Benefits increase at higher pain intensities (38.5% at severe pain) and for challenging populations where expressions are suppressed (37.6% for low expressers). Thermal responses precede visible expressions by 1.2 ± 0.3 seconds, enabling earlier detection. This work was validated on adults only; pediatric applications require dedicated validation. Translation to clinical practice requires multi-site prospective trials, regulatory approval, and careful implementation planning.