Large language model-driven sentiment analysis for facilitating fibromyalgia diagnosis.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Medical Informatics

Background:

• Fibromyalgia (FM) is a complex disorder characterized by widespread pain and emotional distress, presenting diagnostic challenges.
• FM patients exhibit altered cognitive and emotional processing, with a notable attentional bias towards pain-related information.
• This bias can impair cognitive functions like inhibitory control, impacting emotion management and expression.

Purpose of the Study:

• To investigate the utility of open-source large language model (LLM)-driven sentiment analysis in aiding fibromyalgia diagnosis.
• To assess if specific prompt engineering targeting FM-associated language nuances enhances diagnostic accuracy.

Main Methods:

• Enrolled 40 fibromyalgia patients and 40 chronic pain controls.
• Analyzed transcribed patient responses using the Mistral-7B-Instruct-v0.2 LLM.
• Employed two analysis approaches: prompt-engineered for FM nuances and an ablated (un-engineered) approach.
• Calculated diagnostic performance metrics (accuracy, precision, recall, specificity, AUROC) against rheumatologist diagnosis.

Main Results:

• The prompt-engineered LLM approach achieved high performance: 0.87 accuracy, 0.92 precision, 0.84 recall, 0.82 specificity, and 0.86 AUROC.
• The ablated approach yielded lower results: 0.76 accuracy, 0.75 precision, 0.77 recall, and 0.75 specificity.
• The prompt-engineered method demonstrated statistically superior accuracy (McNemar's test p<0.001).

Conclusions:

• LLM-driven sentiment analysis, particularly with prompt engineering, shows potential for facilitating fibromyalgia diagnosis.
• This method can detect subtle linguistic differences in pain expression indicative of FM.
• Further validation, including secondary fibromyalgia patients, is recommended.