Automated MRI Series Classification Across Heterogeneous Datasets Using Limited Manual Annotation

The purpose of this study is to determine the smallest tested fraction of manually labeled training data required for a multimodal artificial intelligence (AI) classifier to achieve satisfactory MRI series classification performance (macro-F1 ≥ 0.80) across five heterogeneous public MRI datasets using digital imaging and communications in medicine (DICOM) metadata, image data, or both. This retrospective study included 3804 examinations and 14,054 MRI series from five public DICOM datasets spanning brain, knee, lumbar spine, liver, and breast MRI, accessed February-April 2026. AI classification models included a random forest classifier using DICOM header metadata, a deep convolutional neural network classifier using the geometric center slice of each MRI series, and decision-fusion models. Models were trained with 5%-40% of studies manually labeled in 5-percentage-point increments, with additional 1%-4% fractions for the knee and lumbar spine. Model performance was evaluated on the remaining studies using study-level splits across five random seeds. Results were summarized as mean ± standard deviation. Zero-label unsupervised clustering results were evaluated separately on the full dataset. The macro-F1 target was reached at the smallest tested training fraction of 1% in the lumbar spine (0.945 ± 0.013), at 2% in the knee (0.818 ± 0.006), at 10% in the brain (0.872 ± 0.027), and at 20% in the liver (0.827 ± 0.016) and the breast (0.847 ± 0.012). The best-performing model differed by dataset, reflecting different contributions from metadata versus image data. Across five separately evaluated public MRI datasets, at least one header-based, image-based, or combined model achieved macro-F1 ≥ 0.80 using no more than 20% manually labeled studies.