Facial Anomaly Appraisal Using Discrepancy Optimization-Driven Automatic Inpainting

This work presents a novel machine learning and signal processing framework designed to consistently detect, localize, and rate facial anomalies such as cleft lip deformity. The goal of this research is to establish a universal and objective measure of facial abnormalities, capable of sensitively identifying both subtle and significant deformities. The proposed model utilizes an enhanced two-phase automatic inpainting method for face normalization, effectively removing anomalies from the image and replacing them with normal facial content. The framework leverages an efficient knowledge distillation model to estimate the initial heatmap that highlights potential facial anomalies. This heatmap is subsequently converted into a mask for inpainting, which is applied to normalize the original face. A deep convolutional neural network (CNN)-based feature extraction method is then employed to compare the anomalous facial image with its normalized counterpart, enabling robust detection and evaluation of various facial anomalies. This is achieved by obtaining a noise-reduced final heatmap that more accurately scores the level of normality in the face. The normalization protocol delivers results comparable to state-of-the-art methods, while being significantly faster, taking less than one second from image upload to obtaining the face rating. This makes it highly feasible for deployment in mobile applications. Additionally, the proposed method does not require anomalous data for model training, while efficiently detecting and assessing various facial anomalies. We demonstrate that this unique computerized image appraisal system generates facial normality/abnormality scores that closely correlate with human intuition, exhibiting 92% correlation with human scores.