Hybrid sensing face detection and registration for low-light and unconstrained conditions

The capability to track, detect, and identify human targets in highly cluttered scenes under extreme conditions, such as in complete darkness or on the battlefield, has been one of the primary tactical advantages in military operations. In this paper, we propose a new collaborative, multi-spectrum sensing method to achieve face detection and registration under low-light and unconstrained conditions. We design and prototype a novel type of hybrid sensor by combining a pair of near-infrared (NIR) cameras and a thermal camera (a long-wave infrared camera). We strategically surround each NIR sensor with a ring of LED IR flashes to capture the "red-eye," or more precisely, the "bright-eye" effect of the target. The "bright-eyes" are used to localize the 3D position of eyes and face. The recovered 3D information is further used to warp the thermal face imagery to a frontal-parallel pose so that additional tasks, such as face recognition, can be reliably conducted, especially with the assistance of accurate eye locations. Experiments on real face images are provided to demonstrate the merit of our method.