Decoding Handwriting Trajectories from Intracortical Brain Signals for Brain-to-Text Communication.

Area of Science:

• Neuroscience
• Biomedical Engineering
• Human-Computer Interaction

Background:

• Clinical brain-computer interfaces (BCIs) have limited ability to decode handwriting trajectories from brain signals.
• Paralyzed individuals attempting handwriting often lack accurate movement labels, causing misalignment issues in BCI decoding.

Purpose of the Study:

• To introduce an innovative decoding framework for brain-computer interfaces (BCIs) to accurately decode handwriting trajectories from neural signals.
• To address shape and temporal distortions between neural activity and movement in BCI decoding.
• To enable effective multi-day data fusion for improved trajectory quality.

Main Methods:

• Recorded intracortical neural signals from a paralyzed individual during attempted handwriting.
• Developed a novel decoding framework to resolve misalignment between neural activity and movement.
• Employed dynamic time warping for translating reconstructed trajectories into text.

Main Results:

• Achieved highly accurate and human-recognizable handwriting trajectory reconstruction, outperforming conventional methods.
• Demonstrated effective multi-day data fusion, enhancing trajectory quality.
• Attained up to 91.1% recognition rate for text translation within a 1000-character database.

Conclusions:

• The novel BCI decoding scheme accurately reconstructs handwriting trajectories from neural signals.
• The framework is applicable to both alphabetic and logographic brain-to-text translation.
• This approach has the potential to revolutionize communication for individuals with motor impairments across diverse languages.