Semantic segmentation model of multi-source remote sensing images was used to extract winter wheat at tillering stage

In complex farmland environments, wheat canopy coverage is insufficient at the tillering stage, posing a considerable challenge to the accurate extraction of its canopy using UAV(unmanned air vehicle) remote sensing images. In this paper, an end-to-end semantic segmentation method based on visible light (RGB) and thermal infrared (TIR) images, Tiff-SegFormer, which fused spectral features and temperature features effectively, is proposed for accurate pixel-level classification of winter wheat at the tillering stage images taken by UAV to segment the wheat canopy and background. Tiff-SegFormer utilizes hierarchical feature representation and efficient self-attention in the encoder stage to extract features of detail contours of RGB images and temperature changes of TIFF images, respectively. In the decoder stage, the features are concatenated and then the channel and spatial attention mechanisms are superimposed, aiming to further improve the segmentation accuracy and efficiency of winter wheat at the tillering stage in UAV remote sensing images. The results show that Tiff-SegFormer can achieve accurate segmentation of wheat canopy and background from UAV images of winter wheat at the tillering stage (mIoU = 84.28%, mPA = 88.97%, accuracy = 94.55%). In order to verify the efficiency of the proposed method, Tiff-SegFormer is compared with four widely used semantic segmentation methods, all of which show better performance. The four methods are UNet, DeepLabv3+, HRNet, SegFormer and four-channel (RGB + TIFF) Segformer. The generalization test shows that the proposed Tiff-SegFormer also achieves better performance than other comparison methods (mIoU = 84.94%, mPA = 91.46%, accuracy = 94.71%). Tiff-SegFormer provides a robust and efficient tool for segmenting winter wheat canopy from UAV remote sensing images of winter wheat at the tillering stage, and has great potential in applications (model implementation and results can be found at https://github.com/wylSUGAR/Tiff-SegFormer ).