Study on multimodal spatially-constrained contrastive learning for knee osteoarthritis severity grading

To address the limitations of single-modal feature coverage and class distribution imbalance in knee osteoarthritis (KOA) classification, this study proposes a Multimodal Spatial-constraint Contrastive Learning (MSCL) model. First, dynamic and static plantar pressure data and human keypoint trajectories are synchronously acquired. The model first feeds dynamic plantar pressure and keypoint data into a multimodal spatial-temporal fusion branch, where graph convolutional networks and Transformers extract spatial-temporal representations of human keypoints and dynamic pressure patterns respectively, followed by Cross Attention fusion. Subsequently, static plantar pressure is processed through a pyramid CNN architecture to generate coarse-grained spatial constraint vectors, which serve as anatomical priors to regularize the fused representations. Finally, a contrastive learning framework is integrated to establish explicit mapping between the enhanced representations and Kellgren-Lawrence (KL) grading system, enabling precise KOA severity stratification. Experimental results demonstrate that the MSCL model achieves 0.94 macro-average accuracy in KL grading, with 7% improvement in F1-scores for imbalanced categories with limited samples. This work establishes a novel paradigm for accurate KOA assessment through multimodal gait analysis.