Related Experiment Video
Updated: Sep 29, 2026
Use of a Video Scoring Anchor for Rapid Serial Assessment of Social Communication in Toddlers
Published on: March 14, 2018
Establishing meaningful score differences for the observer-reported communication ability (ORCA) measure using the
Abigail Rader1, Li Lin2, Nicole Lucas2
1Center for Health Measurement, Department of Population Health Sciences, Duke University School of Medicine, 215 Morris St, Durham, NC, 27701, USA. Abigail.rader@duke.edu.
Purpose:
The Observer-Reported Communication Ability (ORCA) measure assesses communication ability in individuals living with Angelman syndrome (AS). As gene therapies progress through clinical trials, we need to identify what within-person changes in communication are meaningful to families. This study used the Scale Judgment approach to estimate meaningful score differences (MSDs) on the ORCA measure for AS.
Methods:
Caregivers (n = 156) of individuals living with AS completed Scale Judgment tasks showing vignette pairs depicting an individual's communication ability before and after a hypothetical, unspecified treatment. Pairs represented four contexts (lower, higher, expressive, and receptive communication) and score difference magnitudes (2, 4, 6, 8 T-score units). We used an item response theory (IRT) model to estimate a threshold parameter for each vignette pair, then fit a quadratic regression of the parameters on the T-score differences to identify the MSD. ORCA T-scores are calibrated to a mean of 50 and standard deviation of 10 in the AS reference sample.
Results:
Across all contexts, the estimated MSD was 1.80 T-score units (95% CI, 1.11-2.50). Estimates were larger for the lower communication range than higher (2.21 vs. 1.37) and similar for expressive (1.57) and receptive (1.58) communication. When stratified by age, genotype, and baseline score, MSDs varied from 0.82 to 2.58, though confidence intervals overlapped across all comparisons. In exploratory analysis of hypothetical treatment caregivers imagined during the task, estimates were largest among caregivers who reported imagining gene therapy (3.41).
Conclusion:
Caregivers consistently endorsed small score changes as meaningful, suggesting that small treatment effects carry real significance for families. Accounting for variability across contexts and subgroups, we recommend an MSD of 2.0 to 3.5 T-score points on the ORCA measure for individuals with AS, bounded below by the smallest score difference presented to caregivers during the scale judgment task and above by the largest subgroup estimate.
