Bring on the Machines: Could Machine Learning Improve the Quality of Patient Education Materials? A Systematic Search...

Clear and trustworthy information is essential for people who are ill. People with cancer, in particular, are targeted with vast quantities of patient education material, but of variable quality. Machine learning technologies are popular across industries for automated tasks, like analyzing language and spotting readability issues. With the experience of patients with cancer in mind, we reviewed whether anyone has proposed, modeled, or applied machine learning technologies for the assessment of patient education materials and explored the utility of this application.

We systematically searched the literature to identify English-language articles published in peer-reviewed journals or as conference abstracts that proposed, used, or modeled the use of machine learning technology to assess patient education materials. Specifically, we searched MEDLINE, Web of Science, CINAHL, and Compendex. Two reviewers assessed study eligibility and performed study screening.

We identified 1,570 publications in our search after duplicate removal. After screening, we included five projects (detailed in nine articles) that proposed, modeled, or used machine learning technology to assess the quality of patient education materials. We evaluated the utility of each application across four domains: multidimensionality (2 of 5 applications), patient centeredness (1 of 5 applications), customizability (0 of 5 applications), and development stage (theoretical, 1 of 5 applications; in development, 3 of 5 applications; complete and available, 1 of 5 applications). Combining points across each domain, the mean utlity score across included projects was 1.8 of 5 possible points.

Given its potential, machine learning has not yet been leveraged substantially in the assessment of patient education materials. We propose machine learning systems that can dynamically identify problematic language and content by assessing the quality of patient education materials across a range of flexible, customizable criteria. Assessment may help patients and families decide which materials to use and encourage developers to improve materials overall.