A 3D reconstruction method based on multi-views of contours segmented with CNN-transformer for long bones

Area of Science:

• Medical imaging and computer-assisted diagnosis
• Orthopedic treatment and surgical planning
• 3D reconstruction and modeling

Background:

• Traditional 3D imaging (CT, MRI) poses risks like radiation exposure and posture limitations, potentially affecting diagnostic accuracy.
• Existing contour-based methods lack precise segmentation and rely on prior knowledge, limiting their effectiveness in bone reconstruction.

Purpose of the Study:

• To develop an accurate 3D bone reconstruction method using multi-view X-ray contours.
• To introduce a hybrid Convolutional Neural Network (CNN)-Transformer approach for precise bone contour segmentation.

Main Methods:

• A four-step process: bone contour segmentation from X-ray images, 3D point cloud generation (sparse and dense), and surface reconstruction.
• The Trans-DetSeg approach, combining CNN and Transformer, was developed for accurate region detection and bone segmentation.
• Mathematical mapping relationships between multi-view X-ray contours were established for generating point clouds and reconstructing models using the power crust method.

Main Results:

• The Trans-DetSeg approach achieved superior performance with a Dice Similarity Coefficient of 0.949 and Hausdorff Distance of 26.17 on 301 X-ray images.
• 3D reconstruction accuracy was validated in tibia cases, showing reduced Average Surface Distance with increased X-ray views.
• The method demonstrated improved structural information and accuracy compared to CT data.

Conclusions:

• A novel and efficient method for 3D bone reconstruction from multi-view X-ray contours has been successfully developed.
• The hybrid CNN-Transformer approach provides accurate bone contour segmentation, enhancing the overall reconstruction process.
• The proposed method offers a viable, accurate, and potentially safer alternative for orthopedic computer-assisted diagnosis.