When documentation follows the patient: Unprofessional language and behavioral health referrals in substance use...

Substance use disorders (SUDs) are a major public health challenge, and stigma remains a key barrier to care. Unprofessional or stigmatizing language can shape clinician perceptions and affect decision-making. Traditional natural language processing (NLP) often misses context-dependent bias, while large language models (LLMs) pose reliability concerns. This study aimed to (1) develop an LLM-enhanced, human-validated NLP model to detect unprofessional language, (2) quantify unprofessional language and behavioral health referrals, and (3) examine their association among patients with SUD.

In this retrospective cohort study, we analyzed the MIMIC-IV, a large deidentified electronic health record database from a tertiary academic medical center in USA, for adult (≥18 years) with SUD admitted to the emergency department or intensive care unit between 2008 and 2019. A rule-based NLP algorithm detected unprofessional language. Three LLMs (GPT-4, Claude 3, Llama-3) expanded the vocabulary, and expert panel reviewed all generated terms for relevance and clinical realism before integration. Multivariable logistic regression examined the association between unprofessional language and behavioral health referrals.

Of 260,347 patients, 31.5% had SUD. Unprofessional language was more frequent in SUD notes (72% vs. 52%). The LLM-enhanced model improved over baseline (F-score 0.89 to 0.91). Within the SUD cohort, unprofessional language was significantly associated with higher odds of referral (adjusted odds ratio = 1.54) (all p < .001).

Unprofessional language was common in SUD documentation and associated with behavioral health referrals. This human-validated, LLM-enhanced approach highlights how documentation-based stigma may influence care pathways and underscores the need for bias-aware, equitable communication strategies.