A probabilistic approach to realistic face synthesis with a single uncalibrated image

This paper presents a novel approach to automatic face modeling for realistic synthesis from an unknown face image, using a probabilistic face diffuse model and a generic face specular map. We construct a probabilistic face diffuse model for estimating the albedo and normals of the input face. Then, we develop a generic face specular map for estimating the specularity of face. Using the estimated albedo, normal and specular information, we can synthesize the face under arbitrary lighting and viewing directions realistically. Unlike many existing techniques, our approach can extract both the diffuse and specular information of face without involving an intensive 3D matching procedure. We conduct three different experiments to show our improvement over the prior art. First, we compare the proposed algorithm with previous techniques, including the state of the art, to demonstrate our achievement in realistic face synthesis. Moreover, we evaluate the proposed algorithm over non-automatic face modeling techniques through a subjective study. This evaluation is meaningful in that it tells us how far the proposed algorithm as well as others are from the real photograph in terms of the perceptual quality. Finally, we apply our face model for improving the face recognition performance under varying illumination conditions and show that the proposed algorithm is effective to enhance the face recognition rate. Thanks to the compact representation and the effective inference scheme, our technique is applicable for many practical applications, such as avatar creation, digital face cloning, face normalization, deidentification and many others.