Reliability of brain-computer interface language sample transcription procedures.

Area of Science:

• Neuroscience
• Rehabilitation Engineering
• Speech and Language Pathology

Background:

• Brain-computer interfaces (BCIs) offer communication solutions for individuals with severe motor impairments, such as amyotrophic lateral sclerosis (ALS).
• Accurate transcription of BCI-generated communication is crucial for evaluating system performance and user progress.
• Language Activity Monitoring (LAM) logfiles offer a potential method for capturing BCI communication data.

Purpose of the Study:

• To determine the interrater reliability and interjudge agreement for transcribing language samples from daily brain-computer interface (BCI) communication.
• To assess the accuracy and fidelity of using Language Activity Monitoring (LAM) software for recording and transcribing BCI-assisted communication.

Main Methods:

• 345 daily logfile samples from veterans with ALS using a P300-based BCI for augmentative and alternative communication (AAC) were analyzed.
• KeyLAM software recorded communication logfiles during BCI-controlled email and word processing.
• Logfiles were decrypted, transcribed, and analyzed for interrater reliability and interjudge agreement.

Main Results:

• High interrater reliability was found, with 97.6% agreement for total words and 93.5% for total utterances.
• 100% interjudge agreement was achieved for both word and utterance transcription.
• The transcription procedure using LAM data proved accurate across multiple transcribers/raters.

Conclusions:

• Transcribing language samples from LAM data is a highly reliable method for assessing BCI communication.
• LAM tools offer an efficient way to measure language performance, surpassing limitations of audio-visual recordings.
• This reliable transcription process benefits both the BCI research and clinical communities.