Lightness perception in augmented reality: Effects of context, background, motion, and disparity

Lightness constancy-the ability to form a stable percept of surface reflectance despite changes in illumination and context-is critical for the accurate interpretation of visual scenes, including virtual content in optical see-through augmented reality (OST-AR). However, the additive blending of virtual and real-world scenes in AR elevates the luminance of stimuli, which can make it difficult to create a convincing visual impression of low-reflectance surfaces. Here we investigated how a range of visual cues affects lightness perception in OST-AR, with the goal of identifying viewing conditions that expand the range of perceived reflectance for virtual stimuli. In two experiments, participants viewed virtual test patches in a foreground plane and rated their lightness relative to real patches on a paper reference palette. We manipulated four factors: the composition of surrounding elements in the foreground plane, the content and luminance of the background plane, the motion of the background plane, and the relative disparity between foreground and background. We found that the strongest effects on perceived reflectance came from changes to the foreground context: Isolating the test patch or eliminating luminance variation in the surrounding texture substantially reduced the range of perceived reflectance. Surprisingly, even large changes to the background, such as removing texture, introducing motion, or eliminating the depth disparity between foreground and background layers, had much smaller effects. These findings show that lightness perception in OST-AR depends heavily on how the foreground is structured and that including a rich foreground context can help virtual surfaces to generate a wide and realistic range of perceived reflectance.