Decoding Happiness from Neural and Video Recordings for Better Insight Into Emotional Processing in the Brain

Gaining a better understanding of which brain regions are responsible for emotional processing is crucial for the development of novel treatments for neuropsychiatric disorders. Current approaches rely on sparse assessments of subjects' emotional states, rarely reaching more than a hundred per patient. Additionally, data are usually obtained in a task solving scenario, possibly influencing their emotions by study design. Here, we utilize several days worth of near-continuous neural and video recordings of subjects in a naturalistic environment to predict the emotional state of happiness from neural data. We are able to obtain high-frequency and high-volume happiness labels for this task by first predicting happiness from video data in an intermediary step, achieving good results (F₁ = .75) and providing us with more than 6 million happiness assessments per patient, on average. We then utilize these labels for a classifier on neural data (F₁ = .71). Our findings provide a potential pathway for future work on emotional processing that circumvents the mentioned restrictions.