Heuristic Evaluation of a mHealth Diabetes Self-Management System Using Disease Specific Patient Profiles

Patient-centeredness is an important concept in diabetes treatment. We modified Nielsen's expert heuristic evaluation method addressing common gaps: a patient perspective and variability in findings. Two expert, dual-domain evaluators referred to validated patient profiles (mild, moderate, severe diabetes) when conducting uniform evaluation processes on a diabetes mHealth system. Evaluators found 103 usability problems and 224 heuristic violations. For 69% of the problems, the profiles had an effect on severity ratings. "Consistency and Standards" (n = 57) and "Match between the System and Real World" (n = 55) violations dominated at 50%. The overall system severity rating was major. Severity was highest for a severe diabetic profile due to likely visual issues (crowded elements), cognitive concerns (remembering many steps) and for insufficient medication information. Interrater reliability was respectable at Kappa = 0.67. Our novel evaluation method represents one way of improving on a usability expert technique making it more patient-centered with less individual evaluator variability.