Highlights as an Early Predictor of Student Comprehension and Interests

When engaging with a textbook, students are inclined to highlight key content. Although students believe that highlighting and subsequent review of the highlights will further their educational goals, the psychological literature provides little evidence of benefits. Nonetheless, a student's choice of text for highlighting may serve as a window into her mental state-her level of comprehension, grasp of the key ideas, reading goals, and so on. We explore this hypothesis via an experiment in which 400 participants read three sections from a college-level biology text, briefly reviewed the text, and then took a quiz on the material. During initial reading, participants were able to highlight words, phrases, and sentences, and these highlights were displayed along with the complete text during the subsequent review. Consistent with past research, the amount of highlighted material is unrelated to quiz performance. Nonetheless, highlighting patterns may allow us to infer reader comprehension and interests. Using multiple representations of the highlighting patterns, we built probabilistic models to predict quiz performance and matrix factorization models to predict what content would be highlighted in one passage from highlights in other passages. We find that quiz score prediction accuracy reliably improves with the inclusion of highlighting data (by about 1%-2%), both for held-out students and for held-out student questions (i.e., questions selected randomly for each student), but not for held-out questions. Furthermore, an individual's highlighting pattern is informative of what she highlights elsewhere. Our long-term goal is to design digital textbooks that serve not only as conduits of information into the reader's mind but also allow us to draw inferences about the reader at a point where interventions may increase the effectiveness of the material.