Comparative analysis of AI algorithms on real medical data for chronic pain detection

Chronic pain is a pervasive healthcare challenge with profound implications for patient well-being, clinical decision-making, and resource allocation. Traditional detection methods often rely on subjective assessments and manual documentation review, which can be time-consuming and unpredictable. Integrating Artificial Intelligence (AI) into healthcare offers a promising approach to enhance chronic pain management through automated and standardized text analysis. This study examines the use of AI in detecting chronic pain from Italian clinical notes. We leverage machine learning (ML) and natural language processing (NLP) techniques to better understand how chronic pain is documented, thereby enabling efficient, data-driven solutions in nursing and medical practice.

We trained XGBoost, Gradient Boosting (GBM), and BERT-based models (BioBit, bert-base-italian-xxl) on 1,008 annotated Italian clinical notes. Input texts were encoded using TF-IDF, Word2Vec, or FastText for tree-based models and tokenized for transformers. While models were trained on full notes, evaluation was performed on fragmented text to simulate realistic usage. Bayesian optimization and stratified cross-validation over 30 trials ensured robust hyperparameter tuning and performance estimates.

Our AI-based approach achieved high overall accuracy. In particular, XGBoost with TF-IDF embeddings yielded the best performance, reaching an F1-score of 0.92 ± 0.01, with precision at 94%, sensitivity at 91%, and specificity at 93%. The chronic pain notes contained fewer total words (73.91 vs. 119.86, p = 0.0021) and unique words (57.27 vs. 92.78, p = 0.0006) than non-chronic pain notes, underscoring the significance of concise, keyword-rich clinical documentation.

Our findings demonstrate the effectiveness of AI in identifying chronic pain cases from fragmentary clinical notes. By focusing on concise, keyword-oriented text, this work establishes a solid baseline for domain-specific NLP approaches in healthcare. The proposed method reduces the burden of manual review, facilitates real-time decision support, and may standardize chronic pain assessment processes. Furthermore, we plan to explore new embedding techniques specifically designed for short, context-limited clinical notes, where dynamic contextual models (e.g., BERT) often encounter challenges due to insufficient extended textual context.