Improving pose accuracy and geometry in neural radiance field-based medical image synthesis

Neural radiance field (NeRF) models have garnered significant attention for their impressive ability to synthesize high-quality novel scene views from posed 2D images. Recently, the MedNeRF algorithm was developed to render complete computed tomography (CT) projections from a single or a few x-ray images further. Despite this advancement, MedNeRF struggles with accurate pose reconstruction, crucial for radiologists during image analysis, leading to blurry geometry in the generated outputs.

Motivated by these challenges, our research aims to address MedNeRF's limitations in pose accuracy and image clarity. Specifically, we seek to improve the pose accuracy of reconstructed images and enhance the generated output's anatomical detail and quality.

We propose a novel pose-aware discriminator that estimates pose differences between generated and real patches, ensuring accurate poses and deeper anatomical structures in generated images. We enhance volumetric rendering from single-view x-rays by introducing a customized distortion adaptive loss function and present the HTDataset, a new dataset pair that better mimics machine-generated x-rays, offering clearer anatomical depictions with reduced noise.

Our method successfully renders images with correct poses and high fidelity, outperforming existing state-of-the-art methods. The results demonstrate superior performance in both qualitative and quantitative metrics.

The proposed approach addresses the pose reconstruction challenge in MedNeRF, enhances the anatomical detail, and reduces noise in generated images. The use of HTDataset and the innovative discriminator structure lead to significant improvements in the accuracy and quality of the rendered images, setting a new benchmark in the field.