Reconstructing Multi-Stroke Characters From Brain Signals Toward Generalizable Handwriting Brain-Computer Interfaces

Area of Science:

• Neuroscience
• Biomedical Engineering
• Human-Computer Interaction

Background:

• Brain-Computer Interfaces (BCIs) offer communication for individuals with paralysis.
• Existing English handwriting BCIs struggle with complex, multi-stroke characters.
• Chinese characters present a significant challenge due to their complexity and stroke count.

Purpose of the Study:

• To develop a Chinese handwriting BCI system capable of reconstructing multi-stroke trajectories.
• To decode neural signals associated with stroke-writing and pen-lift phases.
• To improve BCI adaptability for complex character sets.

Main Methods:

• Recorded cortical neural signals from the motor cortex during handwriting tasks.
• Developed a stroke-aware approach to individually decode stroke-writing and pen-lift movements.
• Utilized brain signals to reconstruct multi-stroke Chinese character trajectories.

Main Results:

• Identified distinct neural representations for stroke-writing and pen-lift phases.
• Achieved 86% accuracy in reconstructing recognizable Chinese characters from brain signals (400-character set).
• Demonstrated high system stability over a 5-month period.

Conclusions:

• The proposed stroke-aware approach effectively reconstructs multi-stroke handwriting trajectories from neural signals.
• This BCI system shows promise for generalized and adaptable communication for users with paralysis.
• The findings pave the way for advanced BCIs supporting complex writing systems.