The Use of External Anchors When Examining Differences in Scale Performance in Patient Experience Surveys

Area of Science:

• Health Services Research
• Psychometrics
• Survey Methodology

Background:

• Evaluating measurement equivalence is crucial for identifying patient experience disparities.
• External anchors, like scored vignettes, offer a promising but underutilized approach to assess scale use differences.
• Patient surveys are key tools for understanding patient experiences.

Purpose of the Study:

• To demonstrate the utility of vignettes as external anchors for assessing measurement equivalence in patient experience surveys.
• To compare the effectiveness of external anchor analysis versus internal analysis for detecting differential item functioning (DIF).

Main Methods:

• Analyzed UK (General Practice Patient Survey) and US (Consumer Assessment of Healthcare Providers and Systems Clinician and Group) datasets.
• Recruited diverse patient groups: 560 White British, 560 Pakistani adults (UK); 575 Asian American, 505 non-Hispanic White patients (US).
• Patients rated communication quality using established survey questions.

Main Results:

• External anchor analysis revealed substantial evidence of differential item functioning (DIF) in both UK and US data.
• Internal DIF analysis, without an external anchor, yielded minimal evidence of DIF.
• The external anchor method highlighted disparities missed by internal methods.

Conclusions:

• External anchors circumvent the assumption that items lack between-group DIF, which may not hold for patient experience data.
• Patient experience items can be influenced by response tendencies (e.g., extreme responding), potentially masking true DIF.
• Vignette-based external anchors provide a more robust assessment of measurement equivalence for patient experience measures.