Spatio-temporal bidirectional Long Short-Term Memory-based category decoding of natural dynamic facial expression...

Area of Science:

• Affective neuroscience
• Computational neuroscience
• Neuroimaging

Background:

• Facial expression decoding is vital in affective neuroscience.
• Existing models often overlook temporal dynamics and bidirectional information flow in visual processing.
• Accurate decoding requires sophisticated models that integrate spatial and temporal brain data.

Purpose of the Study:

• To develop and validate a spatio-temporal bidirectional long short-term memory (LSTM) model for decoding facial expressions.
• To address limitations of previous models by incorporating temporal dependencies and bidirectional information flow.
• To enhance the accuracy of facial expression decoding from functional magnetic resonance imaging (fMRI) data.

Main Methods:

• Utilized a spatio-temporal bidirectional LSTM model to analyze multi-time fMRI data.
• Simulated time series to capture temporal dependencies within visual cortices.
• Modeled bidirectional information flow between visual regions to account for bottom-up and top-down processing.

Main Results:

• The proposed model achieved significantly higher decoding accuracy for 3 categories of facial expressions compared to baseline and existing models.
• Beta estimates from multi-time response signals (TR 1-6) yielded the best performance.
• The model successfully captured temporal dependencies and bidirectional spatial information from visual cortices.

Conclusions:

• The developed spatio-temporal bidirectional LSTM model offers a superior approach for facial expression decoding.
• Accounting for temporal dynamics and bidirectional information flow enhances decoding performance in affective neuroscience.
• This model advances the understanding of visual processing mechanisms underlying emotion recognition.