The impact of ChatGPT on orthopedic residents' scientific writing: A quasi-experimental study

Scientific writing is essential for orthopedic residents, enabling academic contributions and evidence-based practice. However, challenges such as time constraints, lack of training, and language barriers hinder their writing. AI-based tools like ChatGPT offer potential solutions to improve writing efficiency and quality. This study evaluates the impact of ChatGPT on orthopedic residents' scientific writing by assessing their performance before and after AI assistance in a single cohort.

Thirty-six orthopedic residents first underwent structured training in scientific writing. They then wrote an article within six weeks without AI. In the second phase, they used ChatGPT to write another article over six weeks. Articles were assessed based on six criteria: understanding article writing, identifying research problems, knowledge of research types, understanding research methods, mastering writing techniques, and using correct language.

Following the AI intervention in, writing significantly improved residents' ability to understand article writing (4.3 ± 0.6 vs. 3.1 ± 0.8, p < 0.001), master writing techniques (4.4 ± 0.6 vs. 2.7 ± 0.9, p < 0.001), and use correct language (4.6 ± 0.5 vs. 2.6 ± 1.0, p < 0.001). Following the AI writing intervention, residents had slightly higher scores in identifying research problems, understanding research models, and knowledge of research types, these differences were not statistically significant. Despite improvements in language quality and article structure, AI-generated content lacked originality and critical depth.

ChatGPT enhances writing efficiency and quality but does not replace critical thinking and originality. AI should supplement, not replace, human oversight in scientific writing. Integrating AI-assisted writing into medical education, alongside proper research methodology training, may optimize benefits for orthopedic residents.