RadStructAgent: agentic structuring of free-text chest X-ray reports with small language models

Variability in chest X-ray (CXR) reporting hinders training and evaluation of vision-language models (VLMs) for radiology report generation. Our previous study showed that Structured Radiology Report Generation (SRRG)-style supervision improves VLM-based CXR reporting over free-text supervision. Here, we examine this finding in a larger multi-institutional setting and address a key preprocessing challenge: reliable free-text-to-structured-report conversion using open-source small language models (SLMs).

We evaluate Qwen2.5-VL-3B using free-text and SRRG-style structured supervision across 10 institutions covering 97K samples. We then propose RadStructAgent, an agentic framework that converts free-text CXR reports into structured reports using SLMs, combining LLM-based orchestration, judge feedback, and feedback-guided revision. RadStructAgent is evaluated on 1,000 free-text CXR reports using Qwen3-4B and Qwen3-8B.

For the report generation task, structured supervision improves Qwen2.5-VL-3B over free-text supervision, increasing GREEN from 0.494 to 0.539, with similar gains across other clinical metrics. For the structuring task, RadStructAgent improves both clinical quality (GREEN) and anatomy-level assignment (RadFAA): for Qwen3-4B, from 0.844 to 0.867 and 0.883 to 0.891; for Qwen3-8B, from 0.863 to 0.881 and 0.889 to 0.902.

Structured supervision remains effective for VLM-based CXR report generation when evaluated in a larger multi-institutional setting. RadStructAgent shows that open-source SLMs can be strengthened through agentic verification and refinement, providing a resource-efficient approach for offline and privacy-preserving structured report preparation. Our code, prompt templates, and configuration files are publicly available at: https://github.com/badhon1512/RadStructAgent .