DMP-UNet: a multimodal fusion network for high-precision nitrogen status monitoring in wheat from UAV multispectral...

Achieving high-precision monitoring of wheat nitrogen levels is essential for boosting yield and grain quality, yet the remote sensing community has largely confined itself to single-modal data sources, missing the synergistic benefits of multimodal fusion. To address this, we propose a multimodal feature fusion network that simultaneously utilizes RGB, multispectral, and vegetation index data. The network employs dual-branch encoders to extract modality‑specific features from fused images composed of NDVI, green, and near‑infrared bands; a deformable bidirectional cross‑attention module aligns pixel‑level features and enhances cross‑modal interactions; a modality‑aware pyramid attention feature pyramid network fuses semantic information across scales and modalities; and a dual‑path decoder separates background from wheat regions, with a foreground decoder guided by vegetation attention maps to classify nitrogen status into four levels: Enrichment, Optimum Level, Moderate Nitrogen Deficiency, and Severe Nitrogen Deficiency. Experimental results demonstrate that our method outperforms state‑of‑the‑art models such as DeepLabV3+, U‑Net, U‑Net++, and Attention UNet. Modality contribution analysis confirms the complementary roles of RGB and MS, while ablation studies validate the effectiveness of each key module. These findings confirm the practicality and efficiency of the proposed approach for precise nitrogen monitoring in wheat, highlighting its potential for optimizing fertilization strategies in precision agriculture.