Related Experiment Video
Updated: Aug 29, 2026
Whole-brain Segmentation and Change-point Analysis of Anatomical Brain MRI—Application in Premanifest Huntington's Disease
Published on: June 9, 2018
Multimodal Transformer Fusion of Clinical Information, Medical Text, and Pituitary MRI 2.5D Deep Learning Features
Kaixiang Zhang1, Jiaxu Jia2, Jiancai Lin1
1Department of Orthopaedics, The Second Affiliated Hospital and Yuying Children's Hospital of Wenzhou Medical University, Wenzhou, Zhejiang Province, China (K.Z., J.L., Z.W., L.Y., C.H.).
Rationale And Objectives:
To develop a non-invasive, efficient and accurate auxiliary tool for the precise differential diagnosis between pediatric growth hormone deficiency (GHD) and idiopathic short stature (ISS).
Materials And Methods:
This retrospective two-center study enrolled 618 children as the internal training cohort and 164 children as the independent external test cohort. We constructed and compared 2.5D, 2D and 3D deep learning (DL) models based on pituitary MRI, developed unimodal models for clinical and medical text data, and established a multimodal Transformer fusion (MM_Fusion) model integrating the three modalities, with systematic evaluation of its diagnostic efficacy.
Results:
The 2.5D DL model showed significantly better diagnostic performance and generalization ability than 2D and 3D models. The MM_Fusion model achieved the optimal efficacy, with an AUC of 0.942 in the training cohort and 0.896 in the external test cohort, significantly outperforming all unimodal models, along with excellent calibration performance and clinical utility.
Conclusion:
The pituitary MRI-based 2.5D DL model can effectively differentiate pediatric GHD and ISS. The multimodal Transformer fusion model further improves diagnostic accuracy and generalization, providing a reliable non-invasive auxiliary tool for the precise differential diagnosis of the two diseases.