Related Experiment Video
Updated: May 25, 2025
A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
Published on: January 11, 2020
Medication counseling for OTC drugs using customized ChatGPT-4: Comparison with ChatGPT-3.5 and ChatGPT-4o
Keisuke Kiyomiya1, Tohru Aomori2, Hisakazu Ohtani1,3,4
1Division of Clinical Pharmacy, Keio University Faculty of Pharmacy, Minato-ku, Tokyo, Japan.
Background:
In Japan, consumers can purchase most over-the-counter (OTC) drugs without pharmacist guidance. Recently, generative artificial intelligence (AI) has become increasingly popular. Therefore, medical professionals need to consider the use of generative AI by consumers for medication counseling. We have previously reported responses in Japanese from ChatGPT-3.5 to 264 questions regarding whether each of 22 OTC drugs can be taken under 12 typical patient conditions. The proportion of responses that satisfied the criteria of 1) accuracy, 2) relevance, and 3) reliability with respect to package insert instructions was 20.8%. In November 2023, GPTs were launched, enabling us to construct a customized ChatGPT, using natural language. In the present study, we compared performance in providing medication guidance among a newly customized GPT, the latest non-customized version ChatGPT-4o, and the previous version, ChatGPT-3.5. The aim was to determine whether the customization and version update of ChatGPT improved performance and to evaluate its potential usefulness.
Methods:
We configured customized ChatGPT-4 by executing five instructions in Japanese and uploaded the text of package inserts for 22 OTC drugs as knowledge. We asked the same 264 questions as in our previous study.
Results:
With the customized ChatGPT-4, the percentages of responses that satisfied the criteria of accuracy, relevance, and reliability were 93.2%, 100%, and 60.2%, respectively. Additionally, 56.1% of responses satisfied all three criteria, 2.7-fold higher compared with ChatGPT-3.5 and 1.3-fold higher compared with ChatGPT-4o.
Conclusion:
The performance of our customized GPT far exceeded that of ChatGPT-3.5. In particular, the proportion of appropriate responses to the questions using brand names was significantly improved. ChatGPT can be customized by providing drug package insert information and using appropriate prompt engineering, potentially offering helpful tools in clinical pharmacy.
More Related Videos
Related Concept Videos
01:15Drug Dosage Regimen: Overview
Typically, the starting dose and dosing interval are guided by the manufacturer's recommendations based on clinical trials conducted during and after drug...
01:26Drug Administration and Therapy Phases: Overview
The pharmaceutical phase focuses on leveraging the physicochemical properties of the drug to design and manufacture an effective product. Variants include orally administered tablets or capsules, topical creams or ointments, and parenteral-delivery solutions or emulsions.
The pharmacokinetic phase...
01:02Prescription, Nonprescription and Orphan Drugs
The misuse and addiction to prescription drugs is a growing problem that can affect people of all age groups, specifically teenagers. This can happen when prescription medications are used in ways not intended by the prescriber, such as taking someone else's prescription or using medication for...
01:01Dosage Regimen: Fixed Dose
Fixed-dose regimens can be used for various routes of administration, including intravenous (IV) injections and oral medications. For IV administration, a predetermined amount of the drug is...
01:21Factors Affecting Drug Response: Overview
01:30Combined Effects of Drugs: Antagonism
The most common type is receptor antagonism, where one drug acts as an antagonist to block the effects of another drug by...