Prospective quantitative analysis of hyperparameter and input optimization in GPT-5: comparative contribution to...

Area of Science:

• Artificial Intelligence in Medical Imaging
• Natural Language Processing in Radiology

Background:

• Large language models (LLMs) like GPT-5 show potential for clinical applications.
• Optimizing LLM performance requires careful consideration of input data and model parameters.

Purpose of the Study:

• To evaluate the impact of input format and hyperparameter settings on GPT-5's performance in abdominal case analysis.
• To assess the contribution of GPT-5 assistance to radiologists' diagnostic accuracy and differential diagnosis capabilities.

Main Methods:

• Prospective evaluation of 86 abdominal cases using GPT-5 in browser-based and API-accessed contexts.
• Assessing GPT-5 performance with "only visual" and "visual with imaging findings and clinical presentation" input formats.
• Optimizing GPT-5 using varying temperature and top-p settings, and evaluating its assistance to junior radiologists.

Main Results:

• GPT-5 diagnostic accuracy increased from 12% (visual only) to 58% with additional clinical information (P < 0.001).
• Hyperparameter optimization (temperature: 1.5, top-p: 1) further improved GPT-5 diagnostic accuracy to 73%.
• Radiologists' diagnostic accuracy improved from 71-73% without assistance to 86-94% with GPT-5 assistance.

Conclusions:

• Structured text inputs and API-based hyperparameter optimization enhance GPT-5 performance in abdominal imaging.
• GPT-5 assistance significantly improves diagnostic and differential diagnosis performance for junior radiologists.
• Standardizing hyperparameter settings is crucial for future LLM-based clinical decision-support tools.