Dual Uncertainty-Aware Correspondence Adapting and Retaining for Continual Composed Image Retrieval

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Continual learning research has largely focused on unimodal tasks.
• Multimodal tasks, like Composed Image Retrieval (CIR), remain under-explored in continual learning.
• Real-world retrieval systems face dynamic, ever-changing demands.

Purpose of the Study:

• To establish a novel Continual CIR setting (C2IR) simulating real-world retrieval demands.
• To identify and address key challenges in continual multimodal retrieval: intra-task correspondence uncertainty and inter-task drift uncertainty.
• To propose a Dual Uncertainty-aware Correspondence Adapting and Retaining (U2CAR) framework to tackle these challenges.

Main Methods:

• Introduced the C2IR setting to model dynamic retrieval environments.
• Developed an Uncertainty-based Correspondence Reasoning (UCR) module to manage query-target correspondence uncertainty.
• Designed an Uncertainty-guided Re-parameterization (URep) paradigm to mitigate catastrophic forgetting by consolidating knowledge based on uncertainty.

Main Results:

• The proposed U2CAR framework significantly outperforms existing methods in the C2IR setting.
• Demonstrated robust adaptability to changing retrieval demands.
• Showcased effective mitigation of catastrophic forgetting in continual multimodal learning.

Conclusions:

• The C2IR setting provides a realistic benchmark for continual multimodal retrieval.
• The U2CAR framework effectively addresses uncertainty and forgetting in continual CIR.
• U2CAR offers a promising direction for developing robust and adaptive multimodal continual learning systems.