COMBINER: Composed Image Retrieval Guided by Attribute-Based Neighbor Relations

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Composed Image Retrieval (CIR) aims to locate images using multimodal inputs.
• Existing CIR methods struggle with visually similar images that have distinct attributes, impacting feature fusion and similarity modeling.

Purpose of the Study:

• To address limitations in CIR, particularly the entanglement of attribute semantics, cross-modal inconsistency, and missing supervised signals.
• To propose a unified representation for cross-modal features using attribute prototypes.

Main Methods:

• Introduced COMBINER (COMposed image retrieval network guided By attrIbute-based NEighbor Relations).
• Developed an Adaptive Semantic Disentanglement module for disentangling attribute features.
• Proposed a Unified Prototype-based Composition module for constructing cross-modal unified prototypes (CUP).
• Implemented a Dual Relations Modeling module to mine pairwise and neighbor relations based on attribute similarity.

Main Results:

• COMBINER effectively addresses visually similar but attribute-unrelated samples, a novel approach in CIR.
• The method achieves a more accurate understanding of semantic relations through an attribute prototype-based similarity metric.
• Experiments on three benchmark datasets validate the effectiveness of COMBINER.

Conclusions:

• COMBINER significantly advances Composed Image Retrieval by introducing attribute prototype-based similarity.
• The proposed modules effectively tackle core challenges in cross-modal feature representation and similarity modeling.
• The method demonstrates superior performance in handling complex image retrieval scenarios.