Leveraging large language models in pharmacometrics: evaluation of NONMEM output interpretation and simulation...

Advancements in large language models (LLMs) have suggested their potential utility for diverse pharmacometrics tasks. This study investigated the performance of LLM for generating structure diagrams, publication-ready tables, analysis reports, and conducting simulations using output files from pharmacometrics models. Forty-four NONMEM output files were obtained from the GitHub software repository. The performance of Claude 3.5 Sonnet (Claude) and ChatGPT 4o was compared with two other candidate LLMs: Gemini 1.5 Pro and Llama 3.2. Prompt engineering was conducted for Claude for pharmacometrics tasks such as generating model structure diagrams, parameter tables, and analysis reports. Simulations were conducted using ChatGPT. Claude Artifacts was used to visualize model structure diagrams, parameter tables, and analysis reports. A web-based R Shiny application was implemented to provide an accessible interface for automating pharmacometric model structure diagrams, parameter tables, and analysis reports tasks. Claude was selected for investigation following performance comparisons with ChatGPT 4o, Gemini 1.5 Pro, and Llama on model structure diagram and parameter table generation tasks. Claude successfully generated the model structure diagrams for 40 (90.9%) of the 44 NONMEM output files with the initial prompts, and the remaining were resolved with an additional prompt. Claude consistently generated accurate parameter summary tables and succinct model analysis reports. Modest variability in model structure diagrams generated for replicate prompts was identified. ChatGPT demonstrated simulation capabilities but revealed limitations with complex PK/PD models. LLMs have the potential to enhance key pharmacometrics modeling tasks. However, expert review of the results generated is essential.