Class imbalance aware deep semantic segmentation framework for weed and tobacco crops in UAV imagery

For accurate pesticide application in precision farming, weeds and tobacco plants must be detected to efficiently apply pesticides to weedy areas. There is potential for automated, precise weed and tobacco detection using unmanned aerial vehicle (UAV)-based imaging. Semantic segmentation is a challenge that can be applied to accurately detect weeds in crop field images. Deep learning-based semantic segmentation techniques promise higher accuracy than prior approaches for pixel-level categorization in classical machine learning. In this study, we introduce a novel approach that enhances the accuracy of pixel-level crop-weed interclass classification. We suggested a DeepLabV3Plus ResNeSt model that was trained using the Lovász cross-entropy combined loss and inverse square root frequency weighted class using the tobacco weed UAV-based dataset, achieving a mean average accuracy (aAcc) of 95.93%, a mean intersection over union (mIoU) of 84.99%, and a mean accuracy (mAcc) score of 90.20%. Due to the limited number of pixels identified as weeds, the aerial images used constitute a limited dataset. Therefore, we adjusted class weights using the inverse square root frequency model, which simplified the segmentation process. We observed that the proposed model achieved the highest mean mIoU, indicating that it can accurately detect weeds and tobacco plants.