Retrieval-based adaptive fusion strategy for medical report generation

Retrieval-based medical report generation methods attempt to improve efficiency by reusing historical reports, but their fixed feature-concatenation strategies often introduce cross-case redundancy. Moreover, most methods are designed for low-resolution X-ray images, and their evaluation metrics rely on textual similarity and overlook the implications of misdescription. To address these, we first constructed a high-resolution CT-report dataset, comprising 9 categories of chest CT scans and corresponding reports from 505 patients. Then, we propose RAFS, a retrieval-based adaptive fusion strategy, to dynamically balance contributions from generation and retrieval modules. Finally, we propose DICE, a dual-perspective integrated clinical evaluation including consensus-based positive scoring and penalties of misdescription.

RAFS integrates an attention module to calculate the similarity between the current generated word's hidden state and the retrieved text, passing the result through a fully connected layer to obtain retrieval probabilities. After, obtained attention weights are feed the Sigmoid function and its result for fusing the generation probabilities and retrieval probabilities.

RAFS achieves superior performance with BLEU-4, METEOR, ROUGE_L, CIDEr and the average of DICE scores of 45.8, 32.9, 59.1, 79.3 and 64.6 in the CT report generation task, outperforming existing methods.

RAFS significantly enhances the clinical interpretability of generated reports, with future work dedicated to optimizing the characterization of local pathological lesions.