Assessing ChatGPT's summarization of ⁶⁸Ga PSMA PET/CT reports for patients

Area of Science:

• Medical imaging analysis
• Artificial intelligence in healthcare
• Oncology

Background:

• Artificial intelligence, including large language models like ChatGPT, is increasingly being studied for its potential in medical applications.
• ChatGPT has shown success in responding to medical questions, prompting further investigation into its capabilities with specific medical data.

Purpose of the Study:

• To evaluate the performance of ChatGPT-4 in interpreting structured ⁶⁸Ga PSMA PET/CT reports for newly diagnosed prostate cancer patients.
• To assess the clarity and accuracy of ChatGPT-4's responses regarding cancer staging based on PET/CT findings.

Main Methods:

• 164 structured ⁶⁸Ga PSMA PET/CT reports were inputted into ChatGPT-4.
• ChatGPT-4 was prompted to answer specific questions about local extension, lymph node involvement (regional and distant), bone, and distant organ metastases.
• Responses were scored using a Likert-type scale for clarity and accuracy.

Main Results:

• ChatGPT-4 achieved high mean scores for clarity (4.93-4.99) and accuracy (4.79-4.96) across all assessed parameters.
• Accuracy was significantly lower in patients with distant lymphatic metastases compared to those without (4.28 vs. 4.94, p < 0.001).
• Potentially harmful information was generated in 8% of cases.

Conclusions:

• ChatGPT-4 can successfully interpret structured ⁶⁸Ga PSMA PET/CT reports for prostate cancer staging.
• Physician evaluation remains essential due to the potential for ChatGPT-4 to generate fabricated information.
• Further development is needed before AI models can fully replace expert clinical interpretation.