Beyond averaging: A transformer approach to decoding event related brain potentials

The objective of this study is to assess the potential of a transformer-based deep learning approach applied to event-related brain potentials (ERPs) derived from electroencephalographic (EEG) data. Traditional methods involve averaging the EEG signal of multiple trials to extract valuable neural signals from the high noise content of EEG data. However, this averaging technique may conceal relevant information. Our investigation focuses on determining whether a transformer-based deep learning approach, specifically utilizing attention maps, an essential component of transformer networks, can provide deeper insights into ERP data compared to traditional averaging-based analyses. We investigated the data of an experiment on loudness perception. In the study, 29 normal-hearing participants between 18 and 30 years were presented with acoustic stimuli at five different sound levels between 65 and 95 dB and provided their subjective loudness rating, which was categorized as "too loud" and "not too loud". During the sound presentation, EEG signals were recorded. A convolutional transformer was trained to categorize the EEG data into the two classes ("not too loud" and "too loud"). The classifier exhibited exceptional performance, achieving over 86 % accuracy and an Area under the Curve (AUC) of up to 0.95. Through the utilization of the trained networks, attention maps were generated. Those attention maps provided insights into the time windows relevant for classification within the EEG data. The attention maps above all showed a focus on the time window around 150 to 200 ms, where the average based analysis did not indicate relevant potentials. Employing these attention maps, we were able to gain new perspectives on the ERPs, discovering the attention maps potential as a tool for delving deeper into the analysis of event-related potentials.