Related Experiment Video
Updated: May 2, 2026
04:23
A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images
Published on: April 21, 2023
1.8K
SimpleClick: Interactive Image Segmentation with Simple Vision Transformers
Qin Liu1, Zhenlin Xu1, Gedas Bertasius1
1University of North Carolina at Chapel Hill.
Proceedings. IEEE International Conference on Computer Vision
|September 9, 2024
Summary
SimpleClick is a novel interactive image segmentation method using a plain Vision Transformer (ViT) backbone. It achieves state-of-the-art results with fewer clicks, demonstrating broad applicability.
Area of Science:
- Computer Vision
- Machine Learning
- Artificial Intelligence
Background:
- Interactive image segmentation methods commonly use hierarchical backbones.
- Plain Vision Transformers (ViTs) show promise for dense prediction tasks.
- ViTs offer a foundation model approach adaptable to downstream tasks without backbone redesign.
Purpose of the Study:
- Introduce SimpleClick, the first interactive segmentation method utilizing a plain backbone.
- Explore the effectiveness of plain ViTs in interactive image segmentation.
- Develop a method for efficient object extraction with minimal user interaction.
Main Methods:
- Leveraged a plain, non-hierarchical Vision Transformer (ViT) backbone.
- Introduced a symmetric patch embedding layer for click encoding.
- Utilized a masked autoencoder (MAE) pretraining strategy for the ViT backbone.
Main Results:
- Achieved state-of-the-art performance in interactive image segmentation.
- Obtained 4.15 NoC@90 on the SBD dataset, a 21.8% improvement.
- Demonstrated generalizability on medical imaging datasets.
Conclusions:
- SimpleClick offers a highly effective and efficient approach to interactive image segmentation.
- The method's performance and generalizability highlight its potential as a practical annotation tool.
- Plain backbones are a viable and powerful alternative for interactive segmentation architectures.
