AI Tool Use Among Osteopathic Medical Students: Pilot Digital Diary Study

AI is increasingly integrated into medical education, offering new ways for students to acquire knowledge and support clinical reasoning. However, the extent, patterns, and implications of AI use among medical students remain incompletely understood. Prior studies have relied on retrospective surveys that are susceptible to recall bias and have not quantified AI use as a proportion of total study time.

This pilot study aimed to quantify real-time AI use among medical students, including the proportion of study time devoted to AI, and how use varies by training stage and engagement style (active vs passive). Active use was defined as iterative, bidirectional engagement; passive use was defined as unidirectional consultation with limited interrogation.

This longitudinal observational cohort study recruited medical students from 2 osteopathic medical schools (April-May 2025) to complete a baseline survey and 7 digital diary entries over a 21-day period, delivered via automated SMS every 3 days. Students reported total study time, AI use time, tools used, and purposes of use. The data were analyzed using Stata 19. Multiple linear regression models examined associations between AI use (total minutes and percentage of study time) and key variables, and a mixed-effects model using diary-level data with a random intercept per student addressed within-person variability across entries.

A total of 71 of 1332 (response rate: 5.3%) eligible students completed the baseline survey (mean age 26.6, SD 2.8 y; n=39, 55% identified as men; n=32, 45% identified as women; n=55, 77% preclinical). On average, students reported using AI tools during 19% of their total study time (mean 35.8 of 185.6 min per diary, SD 35.8 min). The most used tool was ChatGPT (n=63, 89%), followed by Google Gemini (n=22, 31%). Clinical-phase students (MS3-MS4) used AI significantly more than preclinical students (MS1-MS2), with an adjusted increase of 19% (P=.003). Students classified as active users spent significantly more total time using AI than passive users (P=.002). Across groups, AI use was primarily passive, including simplifying complex concepts, answering practice questions, and generating summaries. In the multilevel models, preclinical students reported significantly lower AI use than clinical-phase students (P=.02). The intraclass correlation coefficient was 0.47 (95% CI 0.34-0.60).

Although preliminary, these findings suggest that medical students are incorporating AI into a substantial proportion of their study time, with greater use among clinical trainees and active users. Despite this, most use remains passive. Given mixed evidence on AI's impact on deep learning, further research on learning outcomes is needed. Institutions may consider providing guidance on responsible AI use, including critical evaluation and verification of outputs. The digital diary methodology offers a practical approach for capturing real-time AI use and may inform future educational research and intervention design.