A Modular Multi-Agent Reinforcement Learning Framework Guided by Large Language Models: Improving Quality and...

Knowledge-based planning (KBP) frequently requires manual refinement to satisfy institution-specific clinical constraints, while existing automated approaches based on large language models (LLMs) or reinforcement learning (RL) each face distinct limitations in long-term optimization awareness and scalability. We propose a hybrid LLM-guided modular RL framework that integrates the strengths of both approaches for automated lung intensity modulated radiation therapy treatment planning.

The framework employs a 2-level hierarchical architecture in which task-specific RL agents execute dose modifications under the coordination of an LLM supervisory layer comprising Planner, Evaluator, and Supervisor agents. Modular parallel organ-at-risk (OAR), serial OAR, and target coverage agents are trained using state-action-reward-state-action with linear function approximation, with category-level weight sharing enabling cross-OAR generalization from a minimal training set. On achieving full constraint compliance, the system optionally enters Lung Sparing Mode, in which the lung-RL agent continues to reduce lung dose through single-step LLM-supervised iterations. The 2 hybrid configurations using the proposed framework were evaluated on 62 retrospective locally advanced non-small cell lung cancer cases against an institutional KBP model and an LLM-only system using institutional dose-volume constraints.

All 3 LLM-guided systems achieved a 98% clinical goal achievement rate, compared with 74% for KBP alone. Among the 16 cases requiring post-KBP refinement, the hybrid framework reached the clinical goal with 75% fewer LLM supervisory calls than the LLM-only system (1.7 ± 0.7 vs 6.7 ± 5.7) and in less time (20.6 ± 15.7 vs 33.4 ± 32.1 minutes), with the worst case reduced from 109.8 to 58.5 minutes. Enabling Lung Sparing Mode further reduced lung V20 (21.3 ± 8.4% vs 25.2 ± 10.1%; P < .001) and mean lung dose (14.2 ± 4.4 vs 15.0 ± 4.8 Gy; P < .001) relative to Normal mode, at the cost of small increases in spinal cord and esophagus maximum dose that remained well within constraints. The resulting plans achieved lower lung dose than the clinically delivered plans for the same patients (V20 21.3% vs 24.6%; mean 14.2 vs 14.9 Gy).

Integrating modular RL optimization with LLM-based supervision improves both lung sparing and planning efficiency relative to LLM-only automated planning, while enabling scalable deployment through cross-OAR weight sharing with reduced training requirements.