A purpose-based evaluation of information for patients: an approach to measuring effectiveness

To assess the feasibility, internal reliability, and validity, of an assessment tool, purpose-based information assessment (PIA), that we had developed to evaluate how effectively information provided to patients addresses their individual purposes for the information. The study also demonstrated potential novel insight gained by the PIA assessment.

One hundred and eighty-two patients and family members were provided with a booklet on early-stage prostate cancer and its treatment options, in the context of a clinical trial comparing two booklets. Using the PIA, participants rated the importance (4-point Likert scale) of each of six previously identified common purposes for such information: to organize their thoughts, to understand their situation, to decide on treatment, to plan their future, to provide emotional support to others, and to discuss issues. Participants then rated how much their booklet helped address each of their purposes (4-point Likert scale). Evaluations were returned by mail. This report assesses the PIA evaluation of one of the booklets.

One hundred and fifty-six (86%) participants returned evaluations. Participants wanted information for a mean of 5.8 purposes (range 2-7); 72.5% rated the booklet at different levels of helpfulness across their purposes. The assessment showed internal reliability on three constructs tested, and convergent validity on 10 of 11 tested. PIA's individualized purpose-based approach revealed how an overall assessment could be misleading: overall, the booklet was more effective at helping readers decide than at helping them plan (64.7% versus 55.8%, respectively, rated the booklet as "helpful" or better). However, among readers who rated the two purposes as "very important", the booklet had a mean helpfulness rating of 1.95 for deciding compared to a mean of 2.02 for planning. The result suggests that the booklet was not better at helping people decide than at helping them plan, for the readers who most needed the help.

The PIA seems reliable and valid and adequately sensitive. The individualized purpose-based approach to assessing information appears to provide more specific feedback and more insights into its effectiveness than a single, global evaluation.

Developers of information source or educational tools for patients can use an individualized purpose-based assessment, such as the PIA, to identify strengths and limitations of the tools more precisely than global assessments.