A Novel Attention-based Network for Geometry Reconstruction with Error Estimation from Medical Images

Area of Science:

• Medical Imaging
• Computer Vision
• Biomedical Engineering

Background:

• Instance segmentation of anatomical structures is crucial for clinical applications but lacks point-to-point correspondence for atlas-based studies.
• Converting segmentation masks to meshes can introduce errors, necessitating direct mesh generation from medical images.

Purpose of the Study:

• To develop a novel attention-based neural network for direct geometry reconstruction from medical images into high-quality, correspondent meshes.
• To introduce a shape error estimation network for evaluating the reliability of reconstructed geometries.

Main Methods:

• Proposed an attention-based feature extraction network with image and shape self-attention and cross-attention.
• Developed a geometry reconstruction network that deforms a mesh template for correspondence.
• Designed a shape error estimation network to quantify point-to-point errors in reconstructed meshes.

Main Results:

• Achieved higher accuracy and artifact-free results in lumbar spine geometry reconstruction compared to existing methods (UNet++, UTNet, Swin UnetTR, SLT-Net, nnUNet).
• Demonstrated the utility of the shape error estimation network for quality control in clinical settings.

Conclusions:

• The proposed attention-based networks offer a direct and accurate pathway from medical images to correspondent 3D mesh representations.
• The integrated error estimation facilitates reliable clinical application of reconstructed anatomical geometries.