Classifying Drug Ratings Using User Reviews with Transformer-Based Language Models.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Pharmacovigilance

Background:

• Drug review websites offer user-generated text and numeric ratings.
• Numeric ratings may not align with textual sentiment, impacting reliability.
• Automated classification of drug review text can enhance rating accuracy.

Purpose of the Study:

• To develop and compare machine learning models for classifying drug review ratings based on textual content.
• To evaluate the performance of traditional and deep learning models, including transformer networks.
• To identify medical concepts within reviews and analyze their semantic types.

Main Methods:

• Utilized traditional machine learning (Random Forest, Naive Bayesian) with TF-IDF features.
• Implemented transformer-based deep learning models (BERT, Bio_ClinicalBERT, RoBERTa, XLNet, ELECTRA, ALBERT) using raw text.
• Applied Unified Medical Language System (UMLS) concept identification and semantic type analysis.

Main Results:

• The Bio_ClinicalBERT model achieved the highest accuracy at 87%.
• Transformer-based models demonstrated effectiveness in classifying drug reviews solely from text.
• Analysis of UMLS concepts provided insights into review content stratified by predicted sentiment.

Conclusions:

• Transformer models are highly effective for automated drug review classification.
• Text-based classification offers a more reliable alternative to solely relying on numeric ratings.
• This approach can improve the understanding of public drug perception and pharmacovigilance.