Multimodal Fusion of Environmental and Physiological Data for Real-World Personalised Comfort Modelling

Area of Science:

• Environmental Science
• Human-Computer Interaction
• Wearable Technology

Background:

• Most comfort research isolates environmental factors in labs, neglecting real-world multi-domain interactions.
• Understanding personal comfort in everyday environments is crucial for well-being and productivity.

Purpose of the Study:

• To propose and validate a unified Comfort Framework for real-time personal comfort prediction in free-living conditions.
• To integrate diverse data streams including environmental, kinematic, and physiological signals for comfort assessment.

Main Methods:

• A novel framework fusing macro-environmental data (APIs), micro-environmental/kinematic data (smartphone sensors), and physiological data (ECG sensor).
• Real-time comfort state labeling via a consumer smartwatch lap-button protocol.
• Validation through a single-subject pilot study with 18 free-living sessions using Random Forest classification.

Main Results:

• Achieved an F1 macro score of 0.456 ± 0.151 in predicting comfort states, indicating the complexity of free-living conditions.
• Identified significant differences in wrist skin temperature, heart rate, and RMSSD between comfort states.
• SHAP analysis highlighted acoustic features, heart rate variability (HRV) features, and wrist temperature as primary comfort predictors.

Conclusions:

• Real-world comfort prediction using consumer wearables is feasible but more challenging than controlled studies.
• The proposed framework demonstrates potential for multi-domain comfort assessment, with thermal and acoustic signals validated in this pilot.
• Future work should expand validation across more subjects and environmental domains (lighting, air quality).