The Effects of Visual-Olfactory Interactions With Moving Particles on EEG-Based Emotional Classification in AR...

Cross-modal perception, the integration of information from multiple senses, plays a critical role in shaping emotional experiences. This study examines the interactions between visual and olfactory stimuli and their effects on emotional responses, a topic rarely addressed in prior research. Experiments employed five distinct visual stimulation methods that were combined with olfactory stimuli. Participants' emotional responses were assessed via surveys and electroencephalography (EEG) signal analysis. The study varied the color and movement direction of augmented particles to investigate their impact on EEG signals and emotional states. The findings demonstrated significant differences in emotional state classification under the influence of visual-olfactory interactions. Specifically, with backward-moving particles with matching colors (M4), classification accuracy was comparable to that of unimodal olfactory conditions (M1). Other visual stimuli generally caused confusion in classifying emotional responses. The increased valence ratings for pleasant aromas across all visual conditions did not consistently align with EEG-based classification results, suggesting that visual stimuli may introduce complexities into neural signals. These results highlight the intricate dynamics of multisensory interactions, emphasizing the role of visual stimuli in modulating emotional responses. The findings also suggest the potential of visual-olfactory interactions in developing augmented reality (AR) systems. By aligning visual and olfactory cues, AR environments can enhance the user experience and create immersive emotional landscapes, leading to applications for mood modulation and stress relief. This study underscores the relevance of multisensory integration in advancing emotion analysis and affective computing.