Clinical Manifestations.

Area of Science:

• Artificial Intelligence in Medicine
• Natural Language Processing for Healthcare
• Computational Neuroscience

Background:

• Clinical interviews are crucial for dementia diagnosis, guiding further patient evaluation.
• Large Language Models (LLMs) show potential in text analysis but require validation on real medical data.
• Evaluating LLMs for dementia prediction using only clinical notes is an underexplored area.

Purpose of the Study:

• To assess the effectiveness of LLMs in predicting dementia solely from text-based clinical notes.
• To develop and evaluate a novel deep learning classifier for dementia prediction.
• To compare the performance of the proposed model against existing state-of-the-art LLMs.

Main Methods:

• A deep learning classifier was developed using South Korean clinical notes (n=1387) for Alzheimer's Disease (AD) and Normal Control (NC) prediction.
• A hierarchical attention mechanism with Sentence-Level Attention (SLA) and Category-Level Attention (CLA) layers was designed to extract key information from clinical notes.
• The model was trained on 80% and evaluated on 20% of the dataset, with comparisons to baseline LLMs like ChatGPT, LLaMA, and Claude.

Main Results:

• The proposed deep learning model achieved an overall accuracy of 0.74 and an F1-score of 0.72.
• This performance significantly surpassed that of baseline LLMs (ChatGPT, LLaMA, Claude) in robustness and reliability.
• While some LLMs showed strength in specific metrics, their overall results were inconsistent compared to the proposed method.

Conclusions:

• Analyzing clinical notes is a viable and effective strategy for early dementia screening.
• Incorporating text data quality into analysis allows for nuanced contextual information extraction, enhancing prediction performance.
• The developed deep learning approach offers a promising tool for dementia detection in real-world clinical settings.