Modality Synchronization When People With Aphasia Read With Text-to-Speech Support

Text-to-speech (TTS) technology potentially benefits people with aphasia by presenting content through two modalities simultaneously; however, for this to help, eye fixations must synchronize with the auditory rendition of words. Researchers have yet to explore how often and to what extent people with aphasia achieve modality synchronization. This retrospective analysis examined the percent of words people with aphasia see and hear concurrently when reading passages presented via TTS technology. Text-to-speech (TTS) technology potentially benefits people with aphasia by presenting content through two modalities simultaneously; however, for this to help, eye fixations must synchronize with the auditory rendition of words. Researchers have yet to explore how often and to what extent people with aphasia achieve modality synchronization. This retrospective analysis examined the percent of words people with aphasia see and hear concurrently when reading passages presented via TTS technology.

Nine adults with aphasia had their eye movements tracked while processing TTS passages at a preselected default rate of 150 words per minute. Modality synchronization occurred whenever fixation on a written word occurred during the time span beginning 300 ms before auditory presentation and ending at the next word's initiation. Correlations between standardized test scores, unsupported reading rate, and modality synchronization percentages were informative about the association of aphasia and reading impairment severity with achievement of synchronicity.

Three participants demonstrated consistent modality synchronization; average synchronicity ranged from 67% to 76% of passage words. One participant displayed inconsistent synchronization within passages and achieved an average of 58%. The remaining five participants rarely achieved synchronization, with fixations typically lagging substantially behind the auditory presentation. A significant positive correlation occurred between paragraph reading comprehension test scores and modality synchronization percentages.

A default TTS presentation rate does not result in dual modality synchronization for most people with aphasia. This lack of synchronization may contribute to inconsistencies in the benefit people with aphasia experience when provided with TTS support.