Implicit relevance feedback from electroencephalography and eye tracking in image search.

Area of Science:

• Human-computer interaction
• Neuroscience
• Computer science

Background:

• Brain-computer interfacing (BCI) offers direct access to user mental processes, surpassing traditional methods for inferring user information.
• BCI signals can be recorded unobtrusively, eliminating the need for time-consuming explicit user feedback.
• Implicit information from BCI enables real-time user interest profiles for adaptive, personalized software.

Purpose of the Study:

• To explore the potential of implicit relevance feedback using electroencephalography (EEG) and eye tracking.
• To develop and test a demonstrator application simulating an image search engine.
• To assess the effectiveness of BCI in inferring user interest in a human-computer interaction context.

Main Methods:

• Participants queried ambiguous search terms, with one of two interpretations in mind.
• Implicit information from EEG and eye-tracking signals was used to resolve query ambiguity.
• Multivariate classifiers analyzed extracted feature vectors to estimate the intended interpretation of queries.

Main Results:

• The combination of EEG and eye tracking correctly inferred the intended interpretation in 86% of cases.
• Information from EEG and eye tracking modalities proved to be complementary.
• Implicit online feedback from EEG and eye tracking was demonstrated in a simulated realistic use case.

Conclusions:

• BCI methods effectively extract implicit user-related information during human-computer interaction.
• The study highlights the value of complementary EEG and eye-tracking data for inferring user intent.
• This research paves the way for more sophisticated adaptive and personalized software systems.