Pixel-level alignment of facial images for high accuracy recognition using ensemble of patches

The variation of pose, illumination, and expression continues to make face recognition a challenging problem. As a pre-processing step in holistic approaches, faces are usually aligned by eyes. The proposed method tries to perform a pixel alignment rather than eye alignment by mapping the geometry of faces to a reference face while keeping their own textures. The proposed geometry alignment not only creates a meaningful correspondence among every pixel of all faces, but also removes expression and pose variations effectively. The geometry alignment is performed pixel-wise, i.e., every pixel of the face is corresponded to a pixel of the reference face. In the proposed method, the information of intensity and geometry of faces is separated properly, trained by separate classifiers, and finally fused together to recognize human faces. Experimental results show a great improvement using the proposed method in comparison to eye-aligned recognition. For instance, at the false acceptance rate (FAR) of 0.001, the recognition rates are respectively improved by 24% and 33% in the Yale and AT&T datasets. In the labeled faces in the wild dataset, which is a challenging, big dataset, improvement is 20% at a FAR of 0.1.