Universal medical image segmentation via in-context cross-attention

Costin F Ciusdel ¹, Alex Serban ¹, Tiziano Passerini ²

⁰Foundational Technologies, Siemens SRL, Brasov, Romania.

Frontiers in Artificial Intelligence +

|

December 11, 2025

View abstract on PubMed

Summary

This study introduces a universal medical image segmentation method using cross-attention to select relevant support data, improving accuracy and explainability across diverse datasets and modalities.

Area Of Science

Medical Image Analysis
Computer Vision
Artificial Intelligence

Background

Traditional specialist models struggle with adaptation to new medical imaging tasks and distribution shifts.
Generalist pre-trained models and universal segmentation methods offer solutions, with universal approaches excelling in versatility and sample efficiency.
Existing universal methods can be improved by incorporating relevant region pre-selection from support sets.

Purpose Of The Study

To develop a novel universal segmentation method that enhances accuracy by leveraging relevant information from support sets.
To improve the sample efficiency and integration ease of universal segmentation models in medical image annotation pipelines.
To provide an explainable AI approach for medical image segmentation.

Main Methods

Implemented a universal segmentation approach utilizing cross-attention between query and support images.
Introduced an attention up-scaling mechanism for efficient cross-attention computation on multi-resolution features.
Developed an explainability module to visualize relevant support set regions for segmentation outputs.

Main Results

Achieved consistent performance improvements across 29 medical datasets, 9 imaging modalities, and 135 segmentation tasks, even with lightweight models.
Demonstrated proportional gains in segmentation accuracy with increasing support set size, highlighting effective support image selection via cross-attention.
The explainability module showed competitive or superior interpretability compared to established methods like LayerCAM.

Conclusions

The proposed universal segmentation method effectively improves accuracy and explainability in medical image analysis.
Cross-attention and attention up-scaling are key innovations for versatile and efficient medical image segmentation.
The approach offers a promising direction for robust and interpretable AI in healthcare imaging.

Keywords:

deep learning in-context cross attention medical imaging neural networks universal semantic segmentation