A Two-Stage Conditional Diffusion Model With Differential Attention for Hyperspectral and Multispectral Image Fusion

Diffusion models have been used extensively for hyperspectral and multispectral image fusion; however, their intrinsic hallucination phenomenon frequently results in a loss of high-frequency details in the fused images. To address this limitation, this paper proposes a fusion method that integrates a differential attention mechanism with a two-stage conditional diffusion model. The proposed method leverages differential attention to compute the difference between two independent attention maps, thereby producing sparser and more focused attention representations. In addition, the two-stage conditional injection strategy is implemented to realize precise control over the image generation process. In the first stage, feature-level linear modulation via affine transformation is applied within the encoder to maintain global structural consistency. Then, in the second stage, wavelet features extracted from the conditioning images are injected into the decoder to facilitate the restoration of fine-grained details. Extensive validation experiments on the CAVE, Harvard, Pavia Center and Chikusei datasets verify the effectiveness of the proposed method. Compared with numerous state-of-the-art approaches, our method consistently achieves superior performance across key evaluation metrics. On the CAVE dataset, the ERGAS and RMSE metrics improved by 2.16% and 5.01%, and these metrics increased by 1.78% and 1.38% on the Pavia Center dataset, respectively. The code will be available at https://github.com/Ruijie2580/DifferentialDiff.