Distributed computing for the reconstruction of multi-terabyte tomographic X-ray imaging datasets

Area of Science:

• Scientific Imaging
• High-Performance Computing
• Data Processing

Background:

• Tomographic datasets are rapidly increasing in size due to advancements in camera technology and synchrotron light sources.
• Existing reconstruction methods struggle to efficiently process these large-scale datasets on high-performance computing (HPC) systems.

Purpose of the Study:

• To develop and present a novel program for reconstructing large tomographic datasets.
• To enable the distribution of reconstruction processes across CPU- and GPU-powered clusters for enhanced efficiency.

Main Methods:

• Implementation of a parallelized reconstruction program adaptable to both CPU and GPU clusters.
• Testing and validation of the program on a significantly large dataset (24000 pixels wide).

Main Results:

• Successful reconstruction of a 24000-pixel wide tomographic dataset was achieved.
• The program demonstrated efficient data distribution and processing capabilities on HPC systems.
• The developed pipeline is now the default reconstruction method at the DanMAX beamline, MAX IV synchrotron.

Conclusions:

• The presented program offers a scalable and efficient solution for reconstructing large tomographic datasets.
• Its successful implementation at a major synchrotron facility highlights its practical utility and impact on scientific research.
• Reconstructions generated by this pipeline have been published in peer-reviewed journals, validating its performance.