USCNet: Transformer-Based Multimodal Fusion with Segmentation Guidance for Urolithiasis Classification

Area of Science:

• Urology
• Artificial Intelligence
• Medical Imaging

Background:

• Kidney stone disease is a prevalent urological condition requiring accurate stone composition analysis for effective treatment and prevention.
• Current methods rely on postoperative specimens, hindering timely preoperative classification and personalized treatment strategies.

Purpose of the Study:

• To introduce the Urinary Stone Segmentation and Classification Network (USCNet), an AI-driven approach for precise preoperative kidney stone classification.
• To integrate Computed Tomography (CT) imaging with Electronic Health Records (EHR) data for enhanced classification accuracy.

Main Methods:

• Developed USCNet, a Transformer-based multimodal fusion framework incorporating CT-EHR attention and segmentation-guided attention.
• Implemented a dynamic loss function to optimize both segmentation and classification tasks.
• Validated the model on an in-house kidney stone dataset.

Main Results:

• USCNet achieved outstanding performance across all evaluation metrics on the in-house dataset.
• The model's classification efficacy significantly outperformed existing mainstream methods.
• Demonstrated the potential for accurate preoperative kidney stone classification.

Conclusions:

• USCNet offers a promising solution for precise preoperative kidney stone classification, enhancing clinical decision-making.
• The integration of multimodal data (CT and EHR) improves classification accuracy.
• This AI-driven approach has substantial clinical benefits for managing kidney stone disease.