BrainVision: Cross-domain EEG decoding for visual content retrieval and reconstruction

Understanding human visual intent through brain signals remains a fundamental challenge in neuroscience and artificial intelligence. Despite recent advances in brain decoding, existing approaches typically operate within isolated datasets and modalities, limiting their generalization capabilities. This paper introduces BrainVision, a novel framework that bridges visual recognition and emotional EEG datasets to enable comprehensive visual content generation through cross-domain learning. BrainVision addresses the critical challenge of leveraging complementary information across heterogeneous EEG sources by implementing a unified cross-domain alignment strategy. Our framework maps neural patterns from the THINGS-EEG visual recognition dataset and the DEAP emotional response dataset into a shared representation space, enabling three distinct visual output capabilities: (1) accurate content retrieval and classification, (2) detailed linguistic descriptions through adapter-enhanced large language models, and (3) high-fidelity image reconstruction via stable diffusion models. Experimental results demonstrate that BrainVision significantly outperforms single-domain approaches, achieving a 15.3% increase in retrieval accuracy and a 12.7% improvement in structural similarity for reconstructed images compared to state-of-the-art methods. Furthermore, our framework demonstrates robust zero-shot generalization, maintaining 82% of its performance when applied to novel stimuli not seen during training. The multi-modal outputs provide complementary interpretations of neural activity, offering a more comprehensive understanding of visual intent. Our findings establish that integrating diverse neural datasets substantially enhances the capabilities of brain decoding systems, providing a promising direction for developing more intuitive and versatile brain-computer interfaces. BrainVision represents an important step toward bridging the gap between neural activity and rich visual experiences across different cognitive domains.