Baseline Corpus Construction and Proof-of-concept Experiments for AI/ML-Enabled Syndromic Surveillance

The variability in subjective symptom descriptions challenges the early detection of novel diseases and syndromes. This lack of consistent terminology complicates the recognition of emerging health threats, such as Gulf War Illness (GWI), a multisystem condition characterized by chronic, "medically unexplained" symptoms. To address this, standardized symptom corpora are needed to support artificial intelligence and machine learning (AI/ML) tools that can scan electronic health records (EHRs) for early clinical patterns. The purpose of this study was to construct a baseline medical knowledge base (KB) and evaluate if semantic enrichment with diverse synonyms improves automated syndrome detection from clinical notes found in EHRs.

We developed a structured corpus of GWI-related symptoms by mapping hallmark complaints (e.g., fatigue, pain, cognitive difficulties, skin rashes) to International Classification of Diseases, Tenth Revision (ICD-10) codes. To improve coverage beyond canonical codes, we enriched this corpus by constructing 4 KBs: (1) a baseline KB containing only ICD-10 descriptions, (2) a Unified Medical Language System (UMLS)-Enriched KB, (3) a large language model (LLM)-Enriched KB, and (4) a merged KB combining both enrichment strategies. Each KB was used to extract symptom mentions from MIMIC-IV clinical notes via a custom natural language processing (NLP) pipeline. In the evaluation dataset, the label "GWI-positive" was used for patients with evidence of a hospital admission with 3 or more GWI-related ICD-10 codes. The performance of each KB was then compared using standard classification metrics, including precision, recall, F1 score, and the area under the precision-recall curve (AUPRC). All data were de-identified and publicly available; therefore, institutional review board approval was not required.

All enriched KBs outperformed the baseline KB, which achieved an F1 score of 0.628. The UMLS-Enriched KB achieved the highest overall F1 score, reaching a peak F1 score of 0.679 at K = 5, while the merged KB demonstrated the strongest precision and AUPRC. Performance gains peaked at moderate levels of enrichment (K = 3-5), highlighting an optimal range where additional synonyms improved recall without substantially increasing false positives (FPs).

This proof-of-concept study demonstrates that semantic enrichment of a symptom corpus enhances automated detection of GWI in clinical notes. Expanding structured symptom language beyond official codes improves detection sensitivity (recall) while preserving precision, underscoring the importance of corpus enrichment for AI/ML-based syndromic surveillance. A key strength of this work is its transparent, reproducible framework for corpus construction using publicly available data, an approach that can be readily adapted to other syndromes. Limitations include evaluation with a single dataset and reliance on one set of diagnostic codes for labeling, which may underrepresent symptom diversity. Future work will include corpora enrichment techniques that leverage social media data to capture less clinical symptom descriptions and track emerging health threats.