End-to-end learned optics for IR under-display lensless face recognition via digital-twin simulation

Area of Science:

• Computational Imaging
• Biometrics
• Computer Vision

Background:

• Under-display imaging presents challenges for face recognition due to occlusion and reduced optical quality.
• Existing lensless systems often lack robust privacy-preserving mechanisms.
• Deep neural networks (DNNs) show promise for image reconstruction and classification tasks.

Purpose of the Study:

• To develop and evaluate an infrared-assisted, under-display lensless face recognition system.
• To investigate the joint optimization of coded masks and DNN classifiers for improved performance and privacy.
• To compare data-driven mask designs against heuristic approaches.

Main Methods:

• A simulation-based digital twin was used for system evaluation.
• End-to-end optimization of a coded mask and a DNN classifier.
• Comparison of heuristic and learned mask designs under a fixed transmittance constraint.
• Modulation Transfer Function (MTF) analysis to assess frequency response.

Main Results:

• The learned coded mask adapted its frequency response during training, outperforming heuristic designs.
• The optimized system achieved 88.67% accuracy on the DigiFace1M dataset.
• Privacy is enhanced as the mask converts facial images into visually unreadable measurements at capture.

Conclusions:

• Joint optimization of optical elements and DNNs (data-driven optical co-design) is a promising approach for computational imaging.
• Infrared-assisted lensless systems with optimized masks offer a viable solution for privacy-preserving face recognition.
• Learned masks demonstrate superior performance compared to heuristic designs in this lensless recognition task.