Related Experiment Video
Updated: Apr 26, 2026
Characterization of the Sense of Agency over the Actions of Neural-machine Interface-operated Prostheses
Published on: January 7, 2019
Indistinguishable Narratives: A Blinded Comparison of AI- Versus Human-Written Personal Statements in Application to
Henry Avetisian1, Marc A Abdou2, William J Karakash1
1is a Clinical Research Fellow and Medical Student, Department of Orthopaedic Surgery, Keck School of Medicine, University of Southern California, Los Angeles, California, USA.
Background:
As residency program applications become more homogenized due to pass-fail United States Medical Licensing Examination Step 1 scores, pre-clinical grading, and standardized letters of recommendation, personal statements remain a key subjective tool to assess applicant fit. With the rise of large language models like ChatGPT, applicants may increasingly use AI to draft these statements.
Objective:
To determine whether ChatGPT can generate orthopaedic surgery personal statements that are indistinguishable from and comparable in quality to human-written statements.
Methods:
In 2025 at a single urban academic university hospital, ChatGPT-4.0 was given a standardized prompt to generate 5 personal statements across distinct themes. Five de-identified statements from applicants who interviewed or matched at our institution from 2020 to 2021 were randomly selected. The 10 essays were randomized and independently reviewed by 8 blinded assessors, including 6 attending physicians and 2 resident physicians, all of whom serve on the resident admissions committee. Using a 9-question REDCap survey, evaluators rated each statement on readability, originality, authenticity, and overall quality on a 100-point scale; guessed whether each was AI- or human-generated (Yes/No); and indicated whether they would offer an interview (Yes/No). Chi-square tests were used for analysis.
Results:
There were no significant differences between ChatGPT and human-written statements in readability (P=.63), originality (P=.48), authenticity (P=.79), or overall quality (P=.84). Assessors could not reliably identify AI- versus human-written statements (19 of 40 vs 22 of 39, P=.43) and interview offers were similarly comparable (29 of 40 vs 29 of 39, P=.85).
Conclusions:
ChatGPT can produce orthopaedic surgery personal statements that are both indistinguishable from and comparable in quality to human-written ones.
More Related Videos
07:45The Use of Mixed Reality in Custom-Made Revision Hip Arthroplasty: A First Case Report
Published on: August 4, 2022
09:51The Transition to an Anterior-Based Muscle Sparing Approach Improves Early Postoperative Function but is Associated with a Learning Curve
Published on: September 7, 2022