Beyond reading level: a systematic review of the suitability of cancer education print and Web-based materials

Consideration of categories related to reading comprehension--beyond reading level--is imperative to reach low literacy populations effectively. "Suitability" has been proposed as a term to encompass six categories of such factors: content, literacy demand graphics, layout/typography, learning stimulation, and cultural appropriateness. Our purpose was to describe instruments used to evaluate categories of suitability in cancer education materials in published reports and their findings. We searched databases and reference lists for evaluations of print and Web-based cancer education materials to identify and describe measures of these categories. Studies had to evaluate reading level and at least one category of suitability. Eleven studies met our criteria. Seven studies reported inter-rater reliability. Cultural appropriateness was most often assessed; four instruments assessed only surface aspects of cultural appropriateness. Only two of seven instruments used, the suitability assessment of materials (SAM) and the comprehensibility assessment of materials (SAM + CAM), were described as having any evidence of validity. Studies using Simplified Measure of Goobledygook (SMOG) and Fry reported higher average reading level scores than those using Flesh-Kincaid. Most materials failed criteria for reading level and cultural appropriateness. We recommend more emphasis on the categories of suitability for those developing cancer education materials and more study of these categories and reliability and validity testing of instruments.