Concordance of an Artificial Intelligence Model (ChatGPT 4.0) with Physician Decisions in Smoking Cessation Clinics:...

Background: Smoking is one of the leading causes of preventable mortality worldwide. Smoking cessation treatments require personalized therapeutic approaches. Artificial intelligence (AI) is increasingly utilized in clinical decision support systems; however, its role in smoking cessation treatment remains underexplored. This study aims to evaluate the concordance between ChatGPT-4.0-generated treatment recommendations and physician decisions in smoking cessation therapy. Methods: This retrospective and descriptive study was conducted by reviewing the electronic records of patients who presented to a Smoking Cessation Clinic. The ChatGPT-4.0 model was used to compare AI-generated treatment recommendations with physician-prescribed therapies. Concordance rates and the quality of AI-generated information (inappropriate, useful, or perfect information) were assessed. Statistical analyses were performed using SPSS 25.0. Results: A total of 82 patient records were analyzed. The mean age was 40.71 ± 12.87 years (range: 19-69). The overall concordance rate between physicians and ChatGPT-4.0 was 67.1%. Regarding ChatGPT-4.0-generated information quality, 32.9% of cases received inappropriate recommendations, 36.6% received useful recommendations, and 30.5% received optimal recommendations. ChatGPT-4.0 provided inappropriate recommendations in 81.5% of cases involving chronic diseases and 77.8% of cases involving regular medication use (p = 0.021, p = 0.030, respectively). ChatGPT-4.0 achieved the highest rate of optimal recommendations (52.0%) for cytisine therapy. Conclusions: ChatGPT-4.0 can serve as a supportive tool in smoking cessation treatment. However, it remains insufficient in managing complex clinical cases, emphasizing the necessity of physician oversight in final decision-making. Enhancing AI models with larger and more diverse datasets may improve the accuracy of treatment recommendations.