Contextual Semantic Pattern Recognition for Mental Health Monitoring Using ClinicalBERT Transformers

This study aimed to develop an interpretable ClinicalBERT-based framework for mental health monitoring using structured clinical data.

Because the Mental Health Diagnosis and Treatment Monitoring dataset contains structured patient attributes rather than free-text clinical narratives, a structured-to-text transformation module was used to convert the records into contextual descriptions suitable for ClinicalBERT. The proposed model was compared with Random Forest, XGBoost, Bidirectional Encoder Representations from Transformers (BERT), TabTransformer, and a Deep Neural Network, and to strengthen the experimental comparison, contemporary tabular transformer models, including SAINT and FT-Transformer, were also evaluated. Model performance was assessed using accuracy, precision, recall, and F1-score. Cross-validation, confidence interval analysis, statistical significance testing, and ablation experiments were conducted to examine the reliability of the results. SHAP and LIME were used to explain global feature contributions and individual predictions.

ClinicalBERT achieved an accuracy of 90.2%, a precision of 90.5%, a recall of 90.4%, and an F1-score of 90.6%, outperforming all comparison models. The additional validation analyses supported the stability of the results. SHAP and LIME further identified the patient attributes that contributed most strongly to the model's predictions.

The findings indicate that structured clinical records can be transformed into contextual representations for transformer-based mental health monitoring. The framework combines predictive performance with interpretable output, although its evaluation is currently limited to a single dataset. Further validation using multi-institutional and prospective data is required.