Facial color matching in optical see-through augmented reality

Augmented reality (AR) aims to combine elements of the surrounding environment with additional virtual content into a combined viewing scene. Displaying virtual human faces is a widespread practical application of AR technology, which can be challenging in optical see-through AR (OST-AR) because of limitations in its color reproduction. Specifically, OST-AR's additive optical blending introduces transparency and color-bleeding, which is exacerbated especially for faces having darker skin tones, and for brighter and more chromatic ambient environments. Given the increasing prevalence of social AR applications, it is essential to better understand how facial color reproduction is impacted by skin tone and ambient lighting in OST-AR. In this study, a psychophysical experiment was conducted to investigate how participants adjusted colorimetric dimensions of OST-AR-displayed faces to match the color of the same faces viewed on a conventional emissive display. These adjustments were made for faces having six different skin tones, while under different simulated ambient luminance ("low" vs. "high") and chromaticity (warm, neutral, cool). Additionally, participants rated their adjustments for overall appearance match and preference. The results indicate that the magnitude and specific dimensions of colorimetric adjustments needed to make matches varied across skin tones and ambient conditions. The current work is expected to facilitate virtual human face reproduction in AR applications and to foster more equitable and immersive extended reality environments.