A unified multi-task framework enables interpretable chest radiograph analysis

While multimodal deep learning has advanced medical imaging analysis, existing black-box systems may remain confined to isolated tasks, often overlooking the trust-sensitive nature of clinical diagnosis as a multi-task process.

We propose IMT-CXR (interpretable multi-task transformer for chest X-ray analysis), a framework that emulates radiologists' diagnostic workflow through three evidence-driven stages: (1) disease recognition, (2) attribute characterization (e.g., size, location, and severity quantification), and (3) evidence-integrated report generation with traceable decision pathways. The framework employs a unified transformer architecture optimized via medical-domain instruction tuning, which sequentially executes four clinical tasks: multi-label disease classification, lesion localization, anatomical segmentation, and radiology report generation.

Experimental validation demonstrates competitive performance on ten CXR benchmarks under direct inference and fine-tuning settings. In a blinded evaluation of 160 historical reports from four medical centers, three radiologists rated 66% of artificial intelligence (AI)-generated reports as comparable to or surpassing original clinical reports in diagnostic clarity, highlighting the framework's translational potential.

By establishing traceable diagnostic pathways from anatomical findings to conclusions, this work bridges the gap between AI technical metrics and clinical utility, advancing trustworthy AI systems in medical imaging.

This research was partially supported by the Centre for Perceptual and Interactive Intelligence (CPII) Ltd. under the Innovation and Technology Commission (ITC)'s InnoHK.