Psychometric Analysis of an Advance Care Planning Implementation Quality Assessment Tool (ACP-QAT) for Nursing Homes

A reliable and valid assessment of advance care planning (ACP) implementation in nursing homes (NHs) remains a gap, which can be a hindrance for policymakers and researchers to holistically understand the current quality of ACP implementation and identify aspects for improvement. This paper therefore conducted a psychometric analysis of the Advance Care Planning Implementation Quality Assessment Tool (ACP-QAT) to examine its internal consistency reliability, interrater reliability, and construct validity, including convergent validity and known-group validity.

Cross-sectional survey.

An interdisciplinary sample of 31 health care providers (eg, physicians, nurses, and social workers) working in NHs were recruited.

Participants rated the quality of 3 vignettes that represented different ACP implementation quality (high, medium, and low). The primary measure of ACP implementation quality was the tool consisting of 19 binary questions (Yes = 1/No = 0) under 2 dimensions: nursing home structural support, and standardized implementation procedures. A proxy measure was a researcher-constructed 5-point Likert-type item capturing respondents' global impressions of implementation quality assessment.

The tool demonstrated excellent internal consistency reliability for the Structural Support Subscale and Implementation Process Subscale and full scale with Kuder-Richardson Formula 20 coefficients over 0.90. Interrater reliability at the total score level assessed using a 2-way random-effects model intraclass correlation coefficient (ICC) (absolute agreement) was almost perfect (ICC = 0.80). In addition to good indications of reliability, the tool also demonstrated acceptable convergent validity through the strong and positive association between the composite score assessed by the tool and the 1-item global assessment across all vignettes (r = 0.81, P < .001), and known-group validity (χ²(2) = 62.7, P < .001).

These findings suggest that ACP-QAT is a reliable and valid instrument for assessing key components of ACP implementation, including both nursing home structural support and standardized procedures.