Optimizing classification of diseases through language model analysis of symptoms

Area of Science:

• Computational linguistics
• Medical informatics
• Artificial intelligence in healthcare

Background:

• Automating disease prediction from symptoms is crucial for timely medical intervention.
• Traditional methods may lack the sophistication to process complex symptom data.
• Natural Language Processing (NLP) and deep learning offer novel approaches.

Purpose of the Study:

• To evaluate the efficacy of deep learning models, specifically MCN-BERT and BiLSTM, for automated disease prediction.
• To compare the performance of these models when optimized with different hyperparameter tuning methods.
• To assess the models' ability to predict diseases and identify adverse drug reactions (ADRs).

Main Methods:

• Utilized two distinct datasets: Dataset-1 (disease-symptom combinations) and Dataset-2 (Twitter data for ADR identification).
• Employed two Medical Concept Normalization-Bidirectional Encoder Representations from Transformers (MCN-BERT) models and one Bidirectional Long Short-Term Memory (BiLSTM) model.
• Optimized models using hyperparameter tuning methods including AdamP, AdamW, and Hyperopt.

Main Results:

• The MCN-BERT model with AdamP achieved the highest accuracy: 99.58% on Dataset-1 and 96.15% on Dataset-2.
• The MCN-BERT model with AdamW attained 98.33% accuracy on Dataset-1 and 95.15% on Dataset-2.
• The BiLSTM model with Hyperopt demonstrated 97.08% accuracy on Dataset-1 and 94.15% on Dataset-2.

Conclusions:

• Deep learning models, particularly MCN-BERT, exhibit strong potential for accurate disease prediction from symptom descriptions.
• These models can support earlier disease detection, prompt treatment, and enhance remote diagnostic capabilities.
• Further research into NLP and deep learning for medical applications is warranted.