Evaluating text messages from primary care clinics using a CLAS rubric

Text messaging campaigns by healthcare providers seem to increase patient engagement and compliance. As an emergent strategy, text messaging presents bidirectional challenges related to health literacy, culture, and linguistic fluency. A numeric rubric was developed to measure text messages' alignment to federally mandated Culturally and Linguistically Appropriate Services (CLAS) and the National Institutes of Health (NIH) Clear Communication standards.

Use a CLAS rubric evaluate health-related text messages prior to or after distribution from healthcare providers to patients. Using standardized outcome ratings, offer feedback to improve text messages and subsequently increase reported positive patient outcomes.

Using the novel rubric, two bilingual researchers separately scored each message and conducted auditing to agree on message scores. The rubric considered cultural relevance, accessibility, linguistic accessibility, literacy level appropriateness, and clear action-oriented messaging. Each element was awarded a score out of 4 points, with 20 as a perfect score.

Sixty-one outbound text messages were evaluated as part of a comprehensive evaluation for an urban health department health literacy campaign. Text message evaluation was conducted twice during the two-year campaign, with recommendations integrated during the project's second year. Nearly a third of text messages (31.5%) during the project's second year scored a 3.5 out of 4 or higher.

The campaign was generally consistent with NIH best practices and CLAS standards. Application of the rubric and scores before dissemination was most impactful for Federally Qualified Health Centers to improve the quality of health-related text messages.