A memory based model for cartilage and meniscus segmentation in 3D knee MRI

Accurate morphometric assessment of cartilage-such as thickness and volume-via MRI is essential for monitoring knee osteoarthritis. However, segmenting cartilage remains challenging and dependent on extensive expert-annotated datasets, which are heavily subjected to inter-reader variability. Recent advancements in Visual Foundational Models (VFM), particularly memory-based approaches, offer opportunities for improving generalizability and robustness. In this study, we introduce SAMRI-2, a transformer-based deep learning method for cartilage and meniscus segmentation from 3D MRIs using interactive, memory-based VFMs. To improve spatial awareness and convergence, we incorporated a Hybrid Shuffling Strategy (HSS) during training and applied a segmentation mask propagation technique to enhance annotation efficiency. We evaluated SAMRI-2 against four AI models: two 3D convolutional architectures, namely the 3D-VNet and 3D nnU-Net, and two automatic transformer-based models, SAMRI2D and SAMRI3D. These models were trained on 575 3D knee MRI volumes from 270 distinct patients, using both public and internal datasets, and tested on 57 external cases with multi-radiologist annotations and diverse acquisition protocols. Model performance was assessed against reference standards using Dice Similarity Coefficient (DSC) and Intersection over Union, with additional morphometric evaluations to further quantify segmentation accuracy. Our SAMRI-2 model, trained with HSS, outperformed all other models, achieving an average absolute improvement of 0.05 in DSC, with a maximum gain of 0.12 for tibial cartilage. It also demonstrated the lowest cartilage thickness errors, reducing discrepancies by up to threefold. Notably, SAMRI-2 maintained high performance with as few as three user clicks per volume, reducing annotation effort while ensuring anatomical precision. This memory-based VFM with spatial awareness offers a novel approach for reliable AI-assisted knee MRI segmentation, advancing deep learning in musculoskeletal imaging.