Content-aware foveated camera for multi-target tracking.

Area of Science:

• Computer Vision
• Optical Engineering
• Image Processing

Background:

• Modern image sensors generate vast amounts of data, often leading to inefficiencies in memory, computation, and perception.
• Indiscriminate data acquisition overwhelms processing capabilities and limits downstream analysis.

Purpose of the Study:

• To develop a content-aware camera system that dynamically optimizes sensing and magnification for multiple regions of interest (ROIs).
• To reduce data volume and computational load without compromising task performance through intelligent data acquisition.

Main Methods:

• Utilized a phase-only spatial light modulator (SLM) as a solid-state, inertia-free component for beam-steering and focusing.
• Implemented a multi-foveated approach with interleaved frames across foveae to maintain context and high ROI refresh rates.
• Developed a single-SLM, single-sensor architecture for dynamic field-of-view (FOV) reconfiguration and content-aware target tracking.

Main Results:

• Demonstrated a prototype system capable of real-time multi-object tracking with dynamic ROI maintenance across multiple viewpoints.
• Successfully reduced data volume by selectively acquiring and processing information in ROIs.
• Preserved wide-field-of-view situational awareness while enhancing ROI detail.

Conclusions:

• The multi-foveated camera offers an efficient method for intelligent data capture, integrating perception tasks into the acquisition loop.
• This approach shifts computational workload from post-processing to intelligent capture, optimizing system performance.
• Provides a general framework for advanced image sensing and processing in dynamic environments.