BrainPrompting: reconstructing facial memory by brain-based generative AI interaction.

Area of Science:

• Neuroscience
• Computer Science
• Artificial Intelligence

Background:

• Generative artificial intelligence (GenAI) excels at image synthesis but struggles to accurately represent human mental imagery.
• Traditional text-based prompting methods for GenAI lack the nuance to capture complex mental representations.
• Brain activity, specifically electroencephalography (EEG), offers an implicit signal for navigating generative models.

Purpose of the Study:

• To evaluate the accuracy of BrainPrompting in reconstructing human mental images.
• To determine the alignment between generative neural network models and human memory representations.
• To investigate if aggregating neural signals across multiple users enhances image reconstruction accuracy.

Main Methods:

• A neuroadaptive method, BrainPrompting, was employed, utilizing implicit EEG signals to guide a generative model.
• Candidate stimuli were presented to users, who responded with EEG-based recognition signals.
• A "police line-up" experiment was conducted, comparing reconstructed faces to target mental images.

Main Results:

• BrainPrompting successfully reconstructed faces that closely resembled human memory representations.
• A strong alignment was demonstrated between the generative model's output and individual mental representations.
• Accuracy of image reconstruction increased significantly when aggregating neural signals from multiple users.

Conclusions:

• BrainPrompting effectively bridges the gap between human mental imagery and GenAI image synthesis.
• The method shows promise for applications requiring accurate visualization of internal representations.
• Collaborative EEG signal aggregation offers a pathway to achieve above-individual accuracy in BrainPrompting.