AI Chatbot Answers for Drug Dosing Adjustments According to Renal Function in Geriatric Patients Using the New...

Preventable medication errors frequently harm elderly individuals, particularly when kidney filtration rates are reduced. Clinicians often struggle to manage complex drug regimens for these vulnerable populations. Artificial intelligence chatbots are increasingly proposed as potential assistants for calculating precise medication adjustments. No prior work had resolved whether these digital tools maintain sufficient accuracy for clinical decision support. That uncertainty drove the development of a standardized assessment framework for evaluating automated medical advice. Existing metrics lacked the specificity required to judge complex pharmacological recommendations generated by large language models. This gap motivated a rigorous investigation into the reliability of chatbot-generated dosing guidance. Researchers needed a robust method to quantify the safety and precision of these machine-generated outputs.

The aim of this study was to evaluate the reliability of artificial intelligence chatbots in providing drug dosing adjustments for geriatric patients. Researchers sought to determine if these models could safely handle complex medication regimens for individuals with impaired kidney function. The investigation addressed the critical need for scientific validation of automated medical advice tools. No prior work had established a standardized scoring system to quantify the quality of such digital outputs. This gap motivated the development of the AQUOS metric to assess accuracy and safety. The study examined whether performance depended on patient health status, language, or medication complexity. Investigators also aimed to identify the potential for patient harm resulting from inaccurate machine-generated recommendations. The project ultimately sought to clarify whether these technologies are ready for deployment in high-risk clinical environments.

Review approach involved a cross-sectional analysis of four distinct large language models. Investigators utilized a standardized query format to test drug dosing recommendations for one hundred elderly patients. The team prompted these models in both English and German to evaluate language-based performance variations. Researchers assessed each model at two separate time points to determine the consistency of the generated advice. The study employed the newly established AQUOS metric to grade the precision of every response. Experts also applied the World Health Organization safety framework to identify potential risks within the provided guidance. This systematic evaluation covered sixteen hundred total interactions to ensure statistical power. The design focused on comparing performance metrics across varying levels of patient health complexity.

Key findings from the literature indicate that chatbot performance scores ranged from -19.0% to 95.2% across the tested models. The primary observation was that accuracy significantly declined as renal function decreased, with ChatGPT showing a correlation of -0.215. Medication complexity also negatively impacted results, specifically for Scite, which demonstrated a correlation of -0.239. The researchers noted that potential harm increased when patient statuses involved both lower kidney function and higher drug counts. English-language prompts yielded scores up to 4.8% higher than those provided in German. The generated advice proved highly reproducible when tested at two independent time intervals. Despite these findings, the overall quality remained insufficient for safe application in clinical settings. The data confirm that these tools struggle to handle the intricacies of geriatric drug management.

The researchers propose that current chatbot performance remains inadequate for deployment in high-risk medical environments. Synthesis and implications suggest that automated tools fail to provide reliable guidance for patients with significant renal impairment. The study demonstrates that increasing clinical complexity consistently degrades the quality of machine-generated pharmacological advice. Authors emphasize that even the most accurate models do not meet the standards required for safe patient management. These findings highlight a persistent vulnerability when relying on artificial intelligence for critical drug dosing decisions. The data indicate that language differences marginally affect output quality, though this does not overcome broader reliability issues. Experts conclude that human oversight remains mandatory for all medication adjustments in geriatric care settings. Future efforts must address these performance limitations before such technology can be safely integrated into clinical workflows.