Test-Retest Reliability of Core Lexicon Analysis

We examined the test-retest reliability of core lexicon analysis for healthy control participants (HCPs) and individuals with aphasia (IWAs) to facilitate research and clinical implementation of discourse analysis.

Transcripts from 50 HCPs and 61 IWAs (from the AphasiaBank database) were used. All participants completed two testing sessions approximately 1 week apart. The AphasiaBank discourse protocol was administered, including a picture scene, two picture sequences, a story retell, and a procedural task. Computerized Language Analysis software was used to score each task for production of previously established core lexicon items. Test-retest reliability was assessed under a generalization theory framework to examine measurement stability, identify the magnitude of differences in core lexicon scores across time for both groups, and examine the contexts for which reliability is sufficient. Finally, minimal detectable change 95 (MDC₉₅) scores are reported for each task and group.

Core lexicon scores demonstrated variable reliability across tasks for HCPs, from poor to good. In contrast, reliability coefficients demonstrated that core lexicon scores had good-to-excellent reliability for IWAs. Averaging multiple stimuli increased reliability, especially for HCPs. There were no substantial practice effects.

Core lexicon scores were sufficiently stable for IWAs to be used in group-level analyses. Additionally, reliability was excellent for longer stimuli or when averaging multiple shorter stimuli for IWA and sufficient for individual use. Using longer stimuli or averaging core lexicon scores across multiple stimuli is recommended when using as a repeated measure for HCPs to track healthy aging and monitor for changes in language ability. Our results, combined with previous research into validity, support the use of core lexicon analysis in clinical practice and research for HCPs and IWAs. Reporting MDC₉₅ scores for each task and group facilitates clinical uptake of core lexicon analysis for use with clients as appropriate.

https://doi.org/10.23641/asha.34113471.