AI-driven troubleshooting for TrueBeam systems: Development and testing of a GPT-4.1 Chatbot

Troubleshooting linear accelerator faults during patient care is time-critical and cognitively demanding. Access to relevant historical information is often slow and experience-dependent. To streamline information retrieval and support decision-making, a TrueBeam troubleshooting chatbot powered by a large language model (LLM) was developed and tested.

Troubleshooting records from five TrueBeam linacs over eight years were extracted from an in-house database. After removing non-UTF-8 characters and normalizing formatting, each issue was stored as a structured text file for retrieval. Files were indexed in a GPT-4.1-based environment, with parameters (e.g., temperature, retrieved chunks) iteratively tuned. Performance was evaluated using standardized questions across domains including recall, real-time troubleshooting, aggregation, safety, and temporal filtering. Four physicists scored responses using a predefined rubric.

A total of 1394 logs (5.4 MB) were indexed, with indexing completed in 16 min. Mean response time was 7.5 ± 2.5 s. The chatbot performed well in retrieving prior events, summarizing institutional experience, and recognizing when information was unavailable. Performance was largely insensitive to temperature and chunk number, except under severely limited retrieval. Weaknesses included occasional procedural misordering, unclear responsibility between physicists and service personnel, verbosity, and inconsistent temporal filtering.

A GPT-4.1-based RAG chatbot can rapidly surface relevant institutional knowledge for linac troubleshooting and may reduce cognitive burden during machine faults. However, important safety and workflow risks remain. Such systems should function as decision-support tools and require explicit guardrails, role definition, and formal risk evaluation prior to broad clinical deployment.