Neighbor-constrained attention for multiple instance learning in whole-slide histopathology

Whole-slide image (WSI) classification is commonly formulated as multiple instance learning (MIL) with only slide-level labels, yet diagnostically relevant evidence is sparse and spatially organized. A persistent challenge is the spatial context dilemma: spatially agnostic MIL may over-attend isolated artifacts, whereas naive global context injection may oversmooth focal lesions or mix unrelated regions. We propose Neighbor-Constrained MIL (NCMIL), a dual-path MIL framework that combines a Nyströmformer global encoder with Neighbor-Constrained Attention (NCA). Rather than merely combining local and global branches, NCMIL introduces three design choices aimed at spatially structured WSI evidence: a fixed physical neighborhood with a hard attention mask to preserve tissue topology, similarity-weighted neighbor aggregation based on frozen tile embeddings to suppress spatially adjacent but morphologically inconsistent tiles, and adaptive local-global fusion to balance microenvironment fidelity with slide-level context. This inductive bias improves spatial coherence without an explicit graph-construction stage. Extensive 5-fold cross-validation on four public benchmarks shows that NCMIL achieves the best overall performance among the evaluated baselines, with absolute gains of up to +1.3 AUC points, +2.2 F1 points, and +2.2 ACC points over the strongest competitor.