Artificial Intelligence in Medical Writing: Subtle Errors and Their Complex Consequences

Area of Science:

• Medical Informatics
• Artificial Intelligence in Medicine

Background:

• Artificial intelligence (AI) shows promise for enhancing manuscript readability.
• However, AI tools like ChatGPT are prone to generating false information (hallucinations) and citing incorrect references.

Purpose of the Study:

• To evaluate if ChatGPT generates medically incorrect statements, particularly subtle but critical errors, from accurate inputs.
• To assess ChatGPT's ability to retrieve appropriate references and identify its limitations in clinical contexts.

Main Methods:

• ChatGPT-5 was prompted with bullet points on placenta percreta, a condition with recent conceptual shifts.
• A deliberate trap was included regarding the management of percreta in secondary-level institutions.
• An additional experiment evaluated ChatGPT's handling of hypercholesterolemia, including familial hypercholesterolemia.

Main Results:

• ChatGPT generated a largely accurate manuscript on placenta percreta but incorrectly suggested management at secondary-level institutions.
• The AI offered a non-definitive caution during the confirmation stage.
• ChatGPT failed to adequately address familial hypercholesterolemia in a separate experiment.

Conclusions:

• AI-generated medical content can contain subtle yet dangerous inaccuracies.
• The risk of AI "hallucinations" necessitates rigorous domain expert review of AI-assisted manuscripts.
• Over-reliance on AI without expert oversight can lead to the dissemination of critical medical misinformation.