Semantic-based sound retrieval by ERP in rapid serial auditory presentation paradigm.

Area of Science:

• Neuroscience
• Information Retrieval
• Auditory Perception

Background:

• The 'semantic gap' hinders semantic-based multimedia retrieval.
• Visual evoked event-related potentials (ERPs) enable robust semantic image retrieval.
• The utility of auditory evoked ERPs for semantic sound retrieval remains unexplored.

Purpose of the Study:

• To investigate the feasibility of semantic-based sound retrieval using auditory evoked ERPs.
• To compare ERP components and single-trial classification performance in specific vs. semantic sound retrieval tasks.
• To assess the potential of brain-computer interfaces (BCIs) for auditory semantic information extraction.

Main Methods:

• Employed the rapid serial auditory presentation (RSAP) paradigm with animal vocalizations as stimuli.
• Recruited eight BCI-naïve participants for target detection tasks.
• Analyzed event-related potential (ERP) components (N2, P3) and single-trial classification performance under two conditions: specific target detection and semantic category detection.

Main Results:

• While ERP component amplitudes and classification performance slightly decreased in the semantic retrieval task, indicating increased difficulty.
• The best participants achieved an area under the receiver operating characteristic curve (AUC) exceeding 0.77 for single-trial ERP detection.
• These results suggest that auditory evoked ERPs can support semantic-based sound retrieval.

Conclusions:

• Semantic-based sound retrieval using auditory evoked ERPs is potentially feasible.
• The study provides evidence for utilizing auditory ERPs in brain-computer interfaces for semantic information processing.
• Further research can optimize methods for enhanced accuracy and broader applications in auditory information retrieval.