AdaptiveClick: Click-Aware Transformer With Adaptive Focal Loss for Interactive Image Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Interactive image segmentation (IIS) reduces annotation time but suffers from interaction ambiguity, hindering segmentation quality.
• Existing research has under-addressed the critical issue of ambiguity in IIS.

Purpose of the Study:

• To introduce AdaptiveClick, a novel framework for IIS that tackles annotation inconsistencies and improves segmentation quality.
• To develop a click-aware transformer with an adaptive focal loss (AFL) for mask- and pixel-level ambiguity resolution.

Main Methods:

• Proposed AdaptiveClick, the first transformer-based, mask-adaptive segmentation framework for IIS.
• Introduced a click-aware mask-adaptive transformer decoder (CAMD) to enhance click and image feature interaction.
• Developed a generalized adaptive focal loss (AFL) for pixel-adaptive differentiation of hard and easy samples.

Main Results:

• AdaptiveClick demonstrated superior performance compared to state-of-the-art methods across nine datasets using a ViT backbone.
• The proposed AFL was shown to generalize Focal and BCE losses.
• The framework effectively resolves mask- and pixel-level annotation inconsistencies.

Conclusions:

• AdaptiveClick offers a significant advancement in IIS by addressing interaction ambiguity.
• The proposed adaptive focal loss provides a robust mechanism for handling varying data distributions in segmentation tasks.
• The framework's effectiveness and generalizability are validated by extensive experimental results.