AI-Powered Drug Classification and Indication Mapping for Pharmacoepidemiologic Studies: Prompt Development and...

Area of Science:

• Artificial Intelligence in Medicine
• Health Informatics
• Computational Pharmacology

Background:

• Pharmacoepidemiologic studies require accurate Anatomical Therapeutic Chemical Classification System (ATC) drug classification within real-world data (RWD).
• Existing drug classification tools are often costly, unreliable, or have restrictive usage terms.
• There is a need for context-aware classification methods that leverage RWD effectively.

Purpose of the Study:

• To establish large language models (LLMs) as an effective assisting technology for drug classification.
• To develop AI prompts that enable LLMs to reason about drug classifications using RWD.
• To demonstrate the superior accuracy, efficiency, and effectiveness of LLM-based classification compared to alternative methods.

Main Methods:

• An LLM prompt was designed to classify aspirin's therapeutic use (analgesic vs. antithrombotic) using chain-of-thought reasoning.
• The prompt was evaluated on 12,294 anonymized daily dose strings from patients in the US and Canada.
• Performance was benchmarked against a Google Programmable Search Engine using regex-based extraction.

Main Results:

• The LLM achieved 92.5% accuracy in classifying drugs, outperforming the search-based algorithm (82.5%).
• Chain-of-thought reasoning effectively distinguished between different therapeutic uses based on dosage information.
• While generally successful, the prompt formulation faced challenges with drugs having multiple complex classifications.

Conclusions:

• GPT-4o provides a cost-effective and accessible method for drug classification from RWD, adhering to terms of service.
• LLMs, particularly with chain-of-thought prompting, can infer drug class based on dosage and context.
• The widespread availability of LLMs empowers research teams to perform large-scale drug classification, crucial for pharmacoepidemiology.