NoctuaXR: Enhancing Video See-Through Perception for XR in Low-Light Environments with Real-Time Ambient-Aware...

The Video See-Through (VST) technology in Extended Reality Head-Mounted Displays (XR HMDs) allows users to perceive the real-world environment while viewing seamlessly integrated virtual content in passthrough mode. This technology has demonstrated significant potential across domains such as industry, healthcare, and intelligent driving. However, the camera of current VST technologies often struggles with visual fidelity, especially in low-light environments. To address this challenge, we introduce NoctuaXR, a lightweight, low-cost, environment-aware enhancement that dynamically adjusts the pixel-level brightness and saturation of the VST view in real time based on ambient light conditions, significantly improving visual quality in low-light scenarios. In a 10 lx environment, which is a very dim level of light, our tests show that NoctuaXR improves the color accuracy of the Baseline by 68.59% and outperforms the current leading, professional XR HMD, the Varjo XR-4 Focal Edition, by 64.26%. Furthermore, a comprehensive evaluation, including a perceptual quality evaluation and a daily task performance evaluation, confirms NoctuaXR's effectiveness. Our method provides users with a significantly improved visual experience, elevating their task efficiency to a level comparable to the Varjo XR-4. We also present compelling real-world use cases-from in-flight entertainment to navigating a dark room without disturbing others-that highlight NoctuaXR's tangible value in making XR technology safer, more convenient, and truly practical for daily life.