Composed Image Retrieval via Explicit Erasure and Replenishment With Semantic Alignment.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Composed image retrieval requires both removing irrelevant image details and adding desired text-specified elements.
• Existing methods often combine these processes, limiting retrieval precision.

Purpose of the Study:

• To propose a novel method for composed image retrieval that explicitly models two subprocesses: detail suppression and text-based replenishment.
• To enhance the accuracy and effectiveness of retrieving images based on combined image and text references.

Main Methods:

• Developed a composed image retrieval method with three core components: Multi-semantic Dynamic Suppression (MDS), Text-semantic Complementary Selection (TCS), and Semantic Space Alignment (SSA).
• MDS suppresses irrelevant semantic features in the reference image.
• TCS selects, enhances, and replenishes semantic features from the text to the image, with SSA aligning multi-modal semantics.

Main Results:

• The proposed method demonstrated superior performance compared to state-of-the-art approaches.
• Experiments were conducted on three benchmark datasets: Shoes, FashionIQ, and Fashion200K, validating the approach's effectiveness.

Conclusions:

• Explicitly modeling the subprocesses of detail suppression and text-based replenishment leads to improved composed image retrieval.
• The novel method offers a more effective strategy for handling complex image-text retrieval tasks.