Multitask learning for automatic detection of meniscal injury on 3D knee MRI

Magnetic resonance imaging (MRI) of the knee is the recommended diagnostic method before invasive arthroscopy surgery. Nevertheless, interpreting knee MRI scans is a time-consuming process that is vulnerable to inaccuracies and inconsistencies. We proposed a multitask learning network MCSNet_att which efficiently introduces segmentation prior features and enhances classification results through multiscale feature fusion and spatial attention modules. The MRI studies and subsequent arthroscopic diagnosis of 259 knees were collected retrospectively. Models were trained based on multitask loss with coronal and sagittal sequences and fused using logistic regression (LR). We visualized the network's interpretability by the gradient-weighted class activation mapping method. The LR model achieved higher area under the curve and mean average precision of medial and lateral menisci than models trained on a single sagittal or coronal sequence. Our multitask model MCSNet_at outperformed the single-task model CNet and two clinicians in classification, with accuracy, precision, recall, F1-score of 0.980, 1.000, 0.952, 0.976 for medial and 0.920, 0.905, 0.905, 0.905 for the lateral, respectively. With the assistance of model results and visualized saliency maps, both clinicians showed improvement in their diagnostic performance. Compared to the baseline segmentation model, our model improved dice similarity coefficient and the 95% Hausdorff distance (HD₉₅) of the lateral meniscus for 2.3% and 0.860 mm in coronal images and 4.4% and 2.253 mm in sagittal images. Our multitask learning network quickly generated accurate clinicopathological classification and segmentation of knee MRI, demonstrating its potential to assist doctors in a clinical setting.