VIPS: Learning-View-Invariant Feature for Person Search.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Unmanned aerial vehicles (UAVs) are crucial for surveillance, capturing multi-perspective data in complex environments.
• Cross-platform person search in UAV imagery faces challenges due to severe viewpoint variations, occlusions, and cluttered backgrounds.

Purpose of the Study:

• To develop a robust person search framework for UAVs that overcomes viewpoint variations and environmental challenges.
• To leverage large vision-language models (VLMs) for improved cross-view person identification.

Main Methods:

• Proposed a novel two-stage framework, view-invariant person search (VIPS), integrating Faster R-CNN with a view-invariant re-Identification (VIReID) module.
• Utilized VLMs for semantic alignment between text-based ID descriptors and visual features.
• Introduced a learnable mask generator for feature purification and view prompts for perspective shift encoding.

Main Results:

• Achieved state-of-the-art performance on benchmark datasets for cross-platform person search.
• Ablation studies confirmed the effectiveness of individual components, including feature purification and view prompts.
• Demonstrated robust cross-view matching through shared semantic embeddings.

Conclusions:

• VIPS framework significantly enhances person search capabilities in UAV surveillance.
• VLM-derived semantic alignment offers a promising direction for addressing viewpoint disparities in UAV applications.
• The study provides insights for developing advanced real-time UAV-based surveillance systems.