Residual modality alleviation and shape guided consistency learning for cross modality identification under complete...

Area of Science:

• Computer Vision
• Artificial Intelligence
• Pattern Recognition

Background:

• Cross-modality person re-identification (Re-ID) matches individuals across different camera types (e.g., infrared and RGB).
• Existing Re-ID methods often require paired cross-modality images, which are difficult to collect due to limited overlapping observation times.
• Complete modality missing (CMM) scenarios, where no cross-modality pairs exist in training data, pose significant challenges due to large appearance variations.

Purpose of the Study:

• To develop an effective intra-modality supervised person Re-ID method for complete modality missing settings.
• To learn modality-invariant representations from cross-modality unpaired data.
• To address the challenge of matching infrared and RGB images without direct cross-modality supervision.

Main Methods:

• A baseline approach utilizing dual-head cross-entropy loss and multi-modality negative loss to manage cross-modality and intra-modality contrast.
• A novel residual modality alleviation network designed to reduce discrepancies between cross-modality representations.
• A shape-guided consistency learning loss to further enhance representation alignment.

Main Results:

• The proposed method demonstrates superior performance in complete modality missing settings.
• Experiments conducted on SYSU-MM01-CMM and RegDB-CMM datasets validate the effectiveness of the approach.
• The method successfully learns modality-invariant features from unpaired cross-modality data.

Conclusions:

• The developed techniques effectively overcome the limitations of paired data collection in cross-modality person Re-ID.
• The proposed method offers a robust solution for person matching in challenging complete modality missing scenarios.
• This work advances the field of person Re-ID by enabling training with readily available unpaired cross-modality data.