TomoGRAF: An X-ray physics-driven generative radiance field framework for extremely sparse view CT reconstruction

Computed tomography (CT) provides high spatial-resolution visualization of 3D structures for various applications. Traditional analytical/iterative CT reconstruction algorithms require hundreds of angular samplings, a condition may not be met practically for physical and mechanical limitations. Sparse view CT reconstruction has been proposed using constrained optimization and machine learning methods with varying success, less so for ultra-sparse view reconstruction. Neural radiance field (NeRF) is a powerful tool for reconstructing and rendering 3D natural scenes from sparse views, but its direct application to 3D medical image reconstruction has been minimally successful due to the differences in photon transportation and available prior information between optic and X-ray.

We develop TomoGRAF to reconstruct high-quality 3D CT volumes using ultra-sparse projections. TomoGRAF has two main novelties pertinent to X-ray physics and CT imaging. First, TomoGRAF's volume rendering module accumulates x-ray material attenuation passing through an object with CT geometry rather than visible light material color and opacity from surface interaction in NeRF. Second, TomoGRAF penalizes the difference between the simulated and ground truth volume during training besides the 2D views, thus significantly improving the prior fidelity.

TomoGRAF is trained on LIDC-IDRI dataset (1011 scans) and evaluated on an unseen in-house dataset (100 scans) of distinct imaging characteristics from training and demonstrates a vast leap in performance compared with state-of-the-art deep learning and NeRF methods.

TomoGRAF provides the first generalizable solution for image-guided radiotherapy and interventional radiology applications, where only one/a few X-ray views are available, but 3D volumetric information is desired.