AI-driven troubleshooting for TrueBeam systems: Development and testing of a GPT-4.1 Chatbot

Area of Science:

• Medical Physics
• Artificial Intelligence in Healthcare
• Radiation Oncology

Background:

• Troubleshooting linear accelerator (linac) faults is critical during patient care and requires rapid access to historical data.
• Current information retrieval methods are often slow and dependent on individual expertise, impacting efficiency and decision-making.

Purpose of the Study:

• To develop and evaluate a large language model (LLM)-powered chatbot for streamlining troubleshooting of TrueBeam linear accelerator faults.
• To improve information retrieval speed and support clinical decision-making during critical incidents.

Main Methods:

• Extracted and structured eight years of troubleshooting records from five TrueBeam linacs.
• Indexed 1394 logs (5.4 MB) using a GPT-4.1-based Retrieval-Augmented Generation (RAG) environment.
• Evaluated chatbot performance on recall, real-time troubleshooting, aggregation, safety, and temporal filtering, with physicist scoring.

Main Results:

• Indexing completed in 16 minutes with a mean response time of 7.5 ± 2.5 seconds.
• The chatbot successfully retrieved prior events, summarized institutional knowledge, and identified unavailable information.
• Identified weaknesses included occasional procedural errors, unclear responsibilities, verbosity, and inconsistent temporal filtering.

Conclusions:

• A GPT-4.1-based RAG chatbot can efficiently provide institutional knowledge for linac troubleshooting, potentially reducing cognitive burden.
• Significant safety and workflow risks necessitate explicit guardrails, defined roles, and formal risk assessment before widespread clinical adoption.
• The chatbot is best positioned as a decision-support tool rather than an autonomous system.