Utilizing Tympanic Membrane Temperature for Earphone-Based Emotion Recognition

Emotion recognition by wearable devices is essential for advancing emotion-aware human-computer interaction in real life. Earphones have the potential to naturally capture brain activity and its lateralization, which is associated with emotion. In this study, we newly introduced tympanic membrane temperature (TMT), previously used as an index of lateralized brain activation, for earphone-based emotion recognition. We developed custom earphones to measure bilateral TMT and conducted two experiments consisting of emotion induction by autobiographical recall and scenario imagination. Using features derived from the right-left TMT difference, we trained classifiers for both four-class discrete emotion and valence (positive vs. negative) classification tasks. The classifiers achieved 36.2% and 42.5% accuracy for four-class classification and 72.5% and 68.8% accuracy for binary classification, respectively, in the two experiments, confirmed by leave-one-participant-out cross-validation. Notably, consistent improvement in accuracy was specific to models utilizing right-left TMT and not observed in models utilizing the right-left wrist skin temperature. These findings suggest that lateralization in TMT provides unique information about emotional state, making it valuable for emotion recognition. With the ease of measurement by earphones, TMT has significant potential for real-world application of emotion recognition.