A Pilot Study of Medical Student Opinions on Large Language Models

Area of Science:

• Medical Education
• Artificial Intelligence in Medicine
• Health Informatics

Background:

• Artificial intelligence (AI) and large language models (LLMs) are gaining prominence in healthcare.
• LLM-based chatbots like ChatGPT are becoming accessible medical resources for students.
• Understanding current LLM utilization and perceptions among medical students is crucial.

Purpose of the Study:

• To investigate how medical students utilize LLM-based tools in their education.
• To examine medical student perceptions of LLM tools' accuracy and utility.
• To assess the future role and integration of AI in medical practice from a student perspective.

Main Methods:

• A cross-sectional survey was conducted among medical students at the University of Florida College of Medicine in January 2024.
• The survey focused on the utilization of AI and LLM tools and perspectives on AI in medicine.
• 102 medical students responded to the survey.

Main Results:

• All respondents were aware of LLM chatbots; 69% used them monthly for medical purposes.
• 77.1% found LLM-generated information accurate, and 80% intended to continue using them.
• Prior AI knowledge and exposure correlated with increased LLM use, information cross-checking, and trust in AI for clinical decision-making.

Conclusions:

• LLM chatbots are integrated into medical students' study habits, patient care, and research.
• Medical students generally perceive LLM chatbots positively and anticipate their future use.
• There is a significant opportunity to integrate AI education into the medical curriculum to prepare students for its evolving role.