Fog/Edge-Aware State Space Models for Multi-Task Chest X-ray Report Generation and Lesion Detection

Area of Science:

• Radiology and Medical Imaging
• Artificial Intelligence in Healthcare
• Computer Vision and Natural Language Processing

Background:

• Current AI systems for radiology face challenges with computational cost and efficiency in modeling long-range dependencies.
• Automating medical report generation and abnormality detection are key areas for AI in radiology.

Purpose of the Study:

• To propose novel state space model (SSM)-based frameworks for enhanced medical report generation and abnormality localization in chest radiographs.
• To address the limitations of existing AI models in terms of computational cost and efficiency.

Main Methods:

• Developed two frameworks: MambaXray-CTL for report generation and MambaXray-MTL for unified report generation and abnormality localization.
• Integrated a lightweight Mamba-based vision encoder with a large language model (LLM) decoder.
• Employed multi-stage contrastive learning to align visual and textual representations.

Main Results:

• MambaXray-CTL achieved state-of-the-art performance on IU X-ray and CheXpertPlus datasets, outperforming Vision Transformer models in efficiency.
• MambaXray-MTL demonstrated effective unified report generation and accurate abnormality localization.
• The proposed methods significantly reduced computational overhead.

Conclusions:

• Combining state space models with contrastive learning provides efficient, interpretable, and deployable AI solutions for chest radiograph analysis.
• The developed frameworks show significant promise in transforming AI applications within diagnostic radiology.