Personal and Contextual Knowledge Driven Robust Multimodal Affect Recognition using Smartwatches

Accurate and robust affect recognition in the wild is challenging using smartwatches due to scarcity of labeled sensor data. Although smartwatches can easily collect additional information such as, personal and contextual attributes related to affective events, the existing models fail to extract useful representations from such information and thus suffer from performance degradation under various settings. To tackle this problem, we present a novel multimodal machine learning framework that utilizes representation from the personal and contextual attributes as well as from limited sensor data. A real-life user study with 19 participants, followed by extensive evaluation shows that our solution outperforms the existing works across various affective tasks and improves the generalizability of the affective models.