Pupil-adaptive neural holography for eyepiece-free near-eye display

In a holographic near-eye display, the limited sampling of the wavefront by the eye pupil can lead to incomplete image reconstruction or increase speckle noise. Current iterative algorithms based on pupil sampling suffer from two primary limitations: first, the passive optimization of the hologram through pupil masks significantly limits the image quality; second, the computing consumption is difficult to meet the requirements of real-time applications. To overcome these challenges, we propose the pupil-adaptive neural holography (PANH). The core of this framework is the pupil-guided phase generator, which is responsible for actively generating spherical wave phases based on pupil parameters. This move aims to highly concentrate information in the current pupil area, effectively alleviating the decline in image quality caused by insufficient sampling information and multi-pupil tradeoffs. Thanks to the integration of a convolutional neural network (CNN), PANH is capable of generating high-fidelity holograms in real time at a rate of 28 FPS. Simulation and experiments show that PANH always maintains superior image fidelity throughout the entire eyebox. These results indicate that PANH can effectively address the challenge of dynamic pupil changes and contribute to the next generation of high-quality holographic near eye display (NED).