Agreement in functional assessment: graphic approaches to displaying respondent effects

The objective of this study was to examine the agreement between respondents of summary scores from items representing three functional content areas (physical and mobility, personal care and instrumental, applied cognition) within the Activity Measure for Postacute Care (AM-PAC). We compare proxy vs. patient report in both hospital and community settings as represented by intraclass correlation coefficients and two graphic approaches.

The authors conducted a prospective, cohort study of a convenience sample of adults (n = 47) receiving rehabilitation services either in hospital (n = 31) or community (n = 16) settings. In addition to using intraclass correlation coefficients (ICC) as indices of agreement, we applied two graphic approaches to serve as complements to help interpret the direction and magnitude of respondent disagreements. We created a "mountain plot" based on a cumulative distribution curve and a "survival-agreement plot" with step functions used in the analysis of survival data.

ICCs on summary scores between patient and proxy report were physical and mobility ICC = 0.92, personal care and instrumental ICC = 0.93, and applied cognition ICC = 0.77. Although combined respondent agreement was acceptable, graphic approaches helped interpret differences in separate analyses of clinician and family agreement.

Graphic analyses allow for a simple interpretation of agreement data and may be useful in determining the meaningfulness of the amount and direction of interrespondent variation.