Multimodal semantic learning for early fault diagnosis in warp knitting machines via masked fault modeling

Warp knitting machines operate under high-speed and continuous working conditions, where minor abnormalities in yarn tension or vibration may quickly evolve into severe defects. Conventional vision-based inspection methods usually detect faults only after visible defects appear, resulting in substantial latency and limited capability for early warning and predictive maintenance. To address this issue, we propose KnitLLM, a multimodal semantic learning framework for early fault diagnosis in warp knitting machines, which integrates sensor sequences and fabric images for proactive anomaly detection. KnitLLM first employs a vector-quantized variational autoencoder to map heterogeneous temporal signals and image features into a unified discrete semantic space, enabling aligned fault primitive representation across modalities. A Transformer-based masked fault modeling strategy is then introduced to learn long-range dependencies and structural priors of the knitting process by reconstructing masked process states. In addition, a diffusion model is used to augment rare fault samples, alleviating class imbalance and improving recognition performance for minority fault categories. Theoretical analysis from the perspectives of manifold representation, quantization error, and attention structure further supports the semantic abstraction and information preservation capabilities of the proposed framework. Experiments on a real-world industrial dataset containing 10,000 h of warp knitting production data demonstrate that KnitLLM consistently outperforms competitive baselines across multiple metrics, achieving an F1-score of 0.961 and a mean detection latency of 124 ms. The results indicate that the proposed method can identify early faults before visible defects emerge, offering a practical solution for intelligent maintenance and fault warning in high-speed textile manufacturing.