Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Nursing Clinical Information System01:27

Nursing Clinical Information System

763
Nursing Clinical Information System (NCIS)
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
763
Methods of Documentation VI: Case Management Model01:15

Methods of Documentation VI: Case Management Model

564
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
564
Clinical Trials: Overview01:11

Clinical Trials: Overview

2.9K
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...
2.9K
Statistical Software for Data Analysis and Clinical Trials01:12

Statistical Software for Data Analysis and Clinical Trials

526
Statistical software is pivotal in data analysis and clinical trials by providing tools to analyze data, draw conclusions, and make predictions. These software packages range from simple data management applications to complex analytical platforms, supporting various statistical tests, models, and simulation techniques. Their significance lies in their ability to handle vast amounts of data with precision and efficiency, enabling researchers to validate hypotheses, identify trends, and make...
526
Clinical Trials01:16

Clinical Trials

6.6K
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
6.6K
Nursing Assessment01:29

Nursing Assessment

7.5K
The two sources for collecting information are primary and secondary. After gathering information, interpretation and validation help to complete the data. The purpose of assessment is to establish data with the initial information, to interpret data about the patient's perceived needs and health problems, and to respond to these problems identified.
The nurse collects all aspects of the patient's health in the initial assessment, establishing priorities for ongoing focused assessments...
7.5K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Theoretical considerations of the Reporter-Interpreter-Manager-Educator Assessment Framework.

Journal of general internal medicine·2025
Same author

Individual heart failure patient variability in nocturnal hypoxia and arrhythmias.

Medicine·2024
Same author

Using Self-Regulated Learning Microanalysis to Examine Regulatory Processes in Clerkship Students Engaged in Practice Questions.

Perspectives on medical education·2023
Same author

Evaluation of an Individualized Learning Plan Template for the Transition to Residency.

Journal of graduate medical education·2023
Same author

Quantifying diagnostic excellence.

Diagnosis (Berlin, Germany)·2022
Same author

Responses of physicians to an objective safety and quality knowledge test: a cross-sectional study.

BMJ open·2021

Related Experiment Video

Updated: Jun 17, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

533

Assessing the Ability of a Large Language Model to Score Free-Text Medical Student Clinical Notes: Quantitative

Harry B Burke1, Albert Hoang1, Joseph O Lopreiato1

  • 1Uniformed Services University of the Health Sciences, Bethesda, MD, 20814, United States, 1 301-938-2212.

JMIR Medical Education
|August 9, 2024
PubMed
Summary
This summary is machine-generated.

Large language models like ChatGPT can accurately score medical students' clinical notes. This artificial intelligence tool demonstrated a significantly lower error rate than human evaluators, advancing medical education.

Keywords:
AIChatGPTLLMartificial intelligenceclinical informationclinical notesfree-text notesgenerative artificial intelligencegenerative pretrained transformerhistory and physical examinationlarge language modelmedical educationmedical studentmedical studentsmedicinenatural language processingpatientpatientsstandardized patients

More Related Videos

A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
07:50

A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts

Published on: September 20, 2018

15.9K
Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
05:56

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Published on: April 14, 2023

2.4K

Related Experiment Videos

Last Updated: Jun 17, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

533
A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
07:50

A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts

Published on: September 20, 2018

15.9K
Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
05:56

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Published on: April 14, 2023

2.4K

Area of Science:

  • Medical Education
  • Artificial Intelligence in Healthcare
  • Natural Language Processing

Background:

  • Medical education emphasizes teaching clinical information skills.
  • Effective feedback on student-generated clinical notes is crucial.
  • Free-text clinical notes are a key component of medical student assessments.

Purpose of the Study:

  • To evaluate ChatGPT 3.5's capability in scoring medical students' free-text history and physical notes.
  • To compare AI-based scoring with human (standardized patient) scoring.
  • To determine the accuracy of generative pretrained transformer models in medical education.

Main Methods:

  • Retrospective, single-institution study involving 168 first-year medical students.
  • Students' free-text notes were scored against a rubric of 85 case elements.
  • Scoring was performed independently by standardized patients and ChatGPT 3.5.

Main Results:

  • ChatGPT achieved an incorrect scoring rate of 1.0%, significantly lower than standardized patients (7.2%).
  • The overall error rate for ChatGPT was 86% lower than for standardized patients.
  • ChatGPT's mean incorrect scoring rate (12) was significantly lower than standardized patients (85; P=.002).

Conclusions:

  • ChatGPT demonstrates superior accuracy in scoring medical students' clinical notes compared to standardized patients.
  • This study is the first to assess a GPT program for scoring standardized patient-based notes.
  • AI tools like GPT are poised to enhance medical education and clinical practice by providing real-time feedback.