RTPhy-ChatBot: A RAG-Based intelligent assistant for radiotherapy physics using LLaMA3 and AAPM reports

Area of Science:

• Medical physics
• Radiotherapy
• Artificial Intelligence

Background:

• Medical physics is vital in radiotherapy, facing challenges due to rapid technological advancements.
• Medical physicists need efficient information integration for decision-making.

Purpose of the Study:

• Developed RTPhy-ChatBot, an intelligent assistant for radiotherapy physics.
• Aimed to create a reliable tool for medical physicists' daily tasks.

Main Methods:

• Constructed knowledge base from American Association of Physicists in Medicine (AAPM) publications.
• Utilized bge-base-en-v1.5 for embedding and Meta-LLaMA3-8B-Instruct for response generation.
• Compared performance with commercial LLMs and evaluated zero-shot CoT reasoning.

Main Results:

• RTPhy-ChatBot showed strong performance in radiotherapy physics questions, scoring 4.0 ± 0.9.
• Achieved comparable scores to commercial LLMs like Gemini and GPT-4o.
• Excelled in quality assurance standards questions, with Rouge scores matching commercial LLMs.

Conclusions:

• RTPhy-ChatBot is an effective assistant for radiotherapy physics, providing accurate, referenced answers.
• Outperformed or matched commercial LLMs in domain-specific tasks.
• Highlights potential of specialized AI assistants in clinical workflows.