Sparse point annotations for remote sensing image segmentation

In the realm of deep learning, fine-grained semantic segmentation of Remote Sensing Images (RSIs) requires densely annotated pixel samples. However, acquiring such precise labels for training often incurs substantial financial and human costs. While point annotations are easier to acquire than pixel-wise annotations, they lack detailed contour information and spatial coverage. To address this issue, we propose the Point-Based Expand Network (PENet) for Remote Sensing Semantic Segmentation (RSSS). PENet leverages dynamic label expansion guided by high-dimensional semantic feature similarity. To compensate for missing structural cues, a dedicated Segment Anything Model (SAM) branch generates supplementary point-based pseudo-labels that help recover object boundaries and sizes. These SAM-generated labels are then used as anchors in the pseudo-generation branch, which dynamically expands supervision signals by evaluating feature-space similarities. This synergistic mechanism allows PENet to progressively refine semantic labels despite sparse supervision. To better capture spatial information across channels, we integrate the Efficient Multi-scale Attention (EMA) module, which enables dynamic label adjustment and enhances self-supervised learning. We validate the effectiveness of the proposed framework through extensive experiments on the Potsdam and Vaihingen datasets. The results demonstrate the strong potential of point annotations for effective and scalable semantic segmentation of RSIs.