Involving patient partners as appraisers of shared decision making: an explanatory sequential mixed-methods study

Patient and public involvement (PPI) is increasingly recognised as a valuable component of health research, yet patients are rarely included in data analysis within shared decision making (SDM) studies. We aimed to assess the feasibility and value of involving patient partner appraisers of SDM using the OPTION⁵ instrument. Specific objectives were to examine interrater reliability between patient partners and researchers, and to explore patient partners' perspectives on observed differences in interrater reliability.

An explanatory sequential mixed-methods design was used. Four patient partners and three researchers used the validated Danish version of OPTION⁵ to independently score 102 audio-recorded oncology consultations from two randomised trials evaluating patient decision aids delivered before versus during consultations. Each consultation was scored by one trained patient partner and two researchers. Scores were converted to a 0-100 scale. Agreement was assessed using two-way mixed-effects, absolute-agreement intraclass correlation coefficients (ICC) for single and average measures. Item-level analyses included unadjusted and Bonferroni-adjusted p-values. Three patient partners participated in a semi-structured focus group interview to discuss differences in interrater reliability and to reflect on their assessments based on lived experience. Interview data were analysed using a primarily deductive thematic approach informed by hermeneutic-phenomenological principles.

The four patient partners appraised 25-27 consultations each. Total OPTION⁵ scores were median 60.0 for patient partners and 52.5 for researchers. Overall single-rater agreement was moderate (ICC = 0.60, 95% CI 0.46-0.71), and good when averaging raters (ICC = 0.75, 95% CI 0.63-0.83). Item-level differences were small. Although item 2 showed a nominal difference before correction for multiple testing, no item-level differences remained statistically significant after Bonferroni correction. Qualitative analysis revealed three overarching themes: emotion-driven scoring (atmosphere, tone, empathy and recognition), navigating the OPTION⁵ instrument (growing confidence and calibration over time), and evolving roles as co-researchers.

Involving patient partners as appraisers of SDM using OPTION⁵ was feasible, and their appraisal ratings showed moderate reliability with researcher ratings. Findings suggest that, while using the same structured OPTION⁵ scoring framework, patient partners may draw on broader relational cues when interpreting observable SDM behaviours in clinical consultations.

Ethical approval was obtained from the Research Ethics Committee at the University of Southern Denmark (approval ID 23/555) for the involvement of patient partners as co-researchers.