SOP: Selective Orthogonal Projection for Composed Image Retrieval

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Intelligent sensor networks generate vast unstructured visual data.
• Efficient retrieval of specific targets based on complex cross-modal user intents is a significant challenge.
• Existing Composed Image Retrieval (CIR) methods struggle with abstract instructions and suffer from feature distribution shifts and semantic erosion.

Purpose of the Study:

• To propose a novel geometry-based Selective Orthogonal Projection Network (SOP) to overcome limitations in existing CIR methods.
• To enhance the accuracy and efficiency of retrieving targets from large-scale visual sensor data streams.
• To address challenges of focus ambiguity and semantic entanglement in cross-modal retrieval.

Main Methods:

• Developed a Selective Focus Recovery module using information entropy to quantify instruction uncertainty and structural consistency regularization to calibrate query features.
• Introduced Orthogonal Subspace Projection and Geometric Composition Fidelity mechanisms employing Gram-Schmidt orthogonalization.
• Decoupled features into a constant visual base and an orthogonal modification increment to restrict semantic modifications.

Main Results:

• The proposed SOP significantly outperforms State-Of-The-Art (SOTA) methods on benchmark datasets (FashionIQ, Shoes, CIRR).
• Demonstrated improved accuracy in retrieving targets based on complex cross-modal queries.
• Showcased the effectiveness of the proposed modules in mitigating feature distribution shifts and semantic erosion.

Conclusions:

• SOP offers a novel and effective solution for efficient large-scale sensor data retrieval and analysis.
• The geometry-based approach provides a robust framework for handling abstract instructions in cross-modal retrieval.
• The method advances the field of intelligent perception by improving the retrieval of targets from massive visual data streams.