Validity of ChatGPT-generated musculoskeletal images

Area of Science:

• Radiology
• Medical Illustration
• Artificial Intelligence in Medicine

Background:

• Effective communication in radiology research relies heavily on accurate visualizations.
• Large Language Models (LLMs) like ChatGPT4 offer potential for automating figure creation.

Purpose of the Study:

• To evaluate ChatGPT4's capability in generating anatomical schematics for musculoskeletal radiology.
• Assessing the anatomical correctness, annotation accuracy, and usability of LLM-generated figures.

Main Methods:

• ChatGPT4 was used to generate coronal illustrations with annotations for six major joints.
• Four prompt variations were tested for each joint.
• Assessments were conducted by four panellists using a 5-point Likert Scale for anatomical correctness, annotation accuracy, aesthetics, and usability.

Main Results:

• ChatGPT4-generated illustrations showed significant limitations in anatomical accuracy and annotation quality.
• All generated figures received below-average ratings for annotation correctness and research paper usability.
• Inter-rater reliability was good across all assessment domains (ICC = 0.61).

Conclusions:

• Current LLMs like ChatGPT4 do not meet the high standards required for musculoskeletal radiology research figures.
• Further development is needed to improve the realism and accuracy of LLM-generated medical illustrations.
• Iterative refinement is crucial for advancing LLM capabilities in scientific visualization.