"Sadness smile" curve: Processing emotional information from social network for evaluating thermal comfort perception

Thermal comfort is a subjective perception, hence conventional evaluation using meteorological factors faces a technical challenge in precise assessment. Human beings have the nature to differentiate expressions of facial emotions when varied thermal environments are perceived. Facial expression scores can be taken as a predictor of perceived thermal comfort which can be precisely assessed using deep learning against physical factors. In this study, a total of 8314 facial photos were obtained from volunteers in 82 parks of 49 cities via social network. Facial expressions were analyzed to happy, sad, and neutral emotion scores using a professional instrument. Temperature-responsive changes in sadness score (SS) can be fit by a U-shaped curve which was called as the 'sadness smile'. The stationary point of second-order derivative was identified to predict the-most-comfort temperature (22.84 °C), across which a tangent line framed the range of comfort temperatures based on two intersections with first-order derivatives (14.62-31.06 °C). Critical temperature points were identified along a positively correlated line of modified temperature-humidity index against increasing temperatures, which were negatively correlated with SS in autumn and winter. The ResNet model was demonstrated to excellently predict emotion-based thermal comfort perceptions in validation set (R² > 0.5). A nation-wide mapping suggested that many cities of Northwest and North China had local environments that can be perceived with comfort assessed by SS against thermal and cooling temperatures in summer and winter, respectively.