Weakly Supervised Composed Object Re-Identification With Large Models.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Existing object re-identification (re-ID) methods preserve identity but lack appearance modification.
• Composed image retrieval (CIR) allows attribute changes but not identity consistency.
• A gap exists in retrieval tasks requiring both identity preservation and specified appearance modifications.

Purpose of the Study:

• Introduce composed object re-identification (CORI), a novel task addressing the limitations of re-ID and CIR.
• Develop a weakly supervised framework for CORI using large language models (LLMs) and visual question answering (VQA).
• Establish benchmark datasets and a baseline model for CORI evaluation.

Main Methods:

• Proposed a weakly supervised framework leveraging LLMs and VQA for automatic description generation.
• Developed a baseline model with shared-weight image encoders, a text encoder, and a compositor module.
• Utilized contrastive, ID, and triplet losses for joint multimodal composition and identity-aware matching.

Main Results:

• The proposed CORI method significantly outperforms adapted baselines from CIR and re-ID.
• Achieved improvements in Rank@1 accuracy on benchmark datasets: RAP (2.1%), Celeb-reID-light (0.8%), VeRi-776 (9.9%), and VRIC (9.5%).

Conclusions:

• The novel CORI task is feasible and effectively addressed by the proposed weakly supervised framework.
• The developed method demonstrates superior performance in simultaneously preserving identity and modifying appearance based on text guidance.
• The established CORI datasets and baseline model pave the way for future research in this area.