Surg-NAT+: negation-aware vision-language refinement for fine-grained surgical understanding

Surgical vision-language foundation models learn generalizable representations from large-scale surgical data and support flexible adaptation to diverse downstream tasks by using text prompts as classifiers. However, their effectiveness is hindered by a critical semantic limitation: These models often struggle to distinguish between positive and negative textual assertions, a capability essential for fine-grained surgical tasks where the target object may occupy only a small and localized region. This limitation weakens the reliability of current state-of-the-art vision-language adaptation methods, leading to degraded performance when prompts require precise semantic discrimination.

We propose Surg-NAT+, a few-shot vision-language adaptation framework designed to enhance foundation models for fine-grained surgical tasks. First, we introduce a negation-aware contrastive objective that explicitly strengthens the model's ability to differentiate between affirmative and negated textual prompts. This objective is incorporated into pretrained models through multi-level adapter fusion (MAF), enabling hierarchical semantic refinement within the text encoder. Second, we propose a fine-grained self-distillation objective that improves visual grounding by enforcing consistency between global representations of image crops and the corresponding local patch embeddings.

We evaluate our approach on the Cholec80 dataset for few-shot surgical tool recognition using foundation models. Surg-NAT+ achieves state-of-the-art performance across all few-shot regimes, consistently surpassing existing baselines by a substantial margin. Qualitative analyses further demonstrate that our method yields more accurate visual grounding of target tools and enhances semantic separability in the learned feature space.

We presented Surg-NAT+, a lightweight negation-aware adaptation framework that substantially improves the fine-grained recognition capabilities of surgical vision-language models in few-shot, multi-label tool recognition settings. Our approach provides an efficient and semantically robust pathway for adapting foundation models to surgical domains. The code is available at https://github.com/Yutongc-ai/fine-tune-SurgVLP .