Affective Interaction with a Virtual Character Through an fNIRS Brain-Computer Interface.

Area of Science:

• Neuroscience
• Human-Computer Interaction
• Affective Computing

Background:

• Affective brain-computer interfaces (BCI) aim to create naturalistic affective interactions.
• Neurofeedback (NF) offers a method to voluntarily regulate brain activity.
• Dorsolateral prefrontal cortex (DL-PFC) asymmetry is linked to approach/avoidance motivations.

Purpose of the Study:

• To explore affective engagement with a virtual agent using a neurofeedback (NF) paradigm.
• To investigate the use of dorsolateral prefrontal cortex (DL-PFC) activity for controlling virtual agent expressions.
• To assess the realism and consistency of virtual agent responses based on user's mental disposition.

Main Methods:

• Functional near infrared spectroscopy (fNIRS) measured left-asymmetric DL-PFC activity as a proxy for approach.
• This brain activity was mapped to control a neural network that activated action units (AUs) for facial expressions.
• An experiment was conducted with 18 subjects interacting with a virtual agent under NF.

Main Results:

• Subjects successfully controlled their mental disposition via NF to engage with the virtual agent.
• Participants perceived the virtual agent's facial expressions as realistic and consistent with their affective state.
• The study confirmed the feasibility of using DL-PFC asymmetry for affective BCI control.

Conclusions:

• Affective BCI, utilizing NF and DL-PFC activity, can facilitate realistic affective interactions with virtual agents.
• This paradigm enables volitional control over brain areas not typically under conscious regulation.
• The research integrates brain metabolic data with a controllable virtual environment for affective communication.