A proof of concept reinforcement learning based tool for non parametric population pharmacokinetics workflow
J D Otalvaro1,2,3, W M Yamada1, A M Hernandez2
1Laboratory of Applied Pharmacokinetics and Bioinformatics, Department of Infectious Diseases, Children's Hospital Los Angeles, Los Angeles, CA, USA.
Journal of Pharmacokinetics and Pharmacodynamics
|December 8, 2022
Summary
Reinforcement learning (RL) automates population pharmacokinetic model building. A SARSA agent successfully optimized models, achieving results comparable to manual methods, demonstrating RL
Area of Science:
- Pharmacometrics
- Machine Learning
- Computational Biology
Background:
- Population pharmacokinetic (PopPK) model building is an iterative process involving model specification, evaluation, and optimization.
- This iterative cycle aligns with Reinforcement Learning (RL) principles, offering potential for automation.
Purpose of the Study:
- To present the conceptual framework and implementation of an RL algorithm for automating PopPK model building.
- To demonstrate how pharmacometricians can utilize RL to streamline the iterative model development process.
Main Methods:
- Developed a discretized state and action space for the RL agent.
- Utilized the SARSA (State-Action-Reward-State-Action) algorithm, configured with 1000 episodes and 30 actions per episode.
- Integrated the SARSA agent with the Non-Parametric Optimal Design algorithm for parameter optimization.
Main Results:
- The RL-based agent achieved comparable model likelihood and support point distribution to the original study.
- Training the RL agent took 5.5 hours, with potential for further time optimization.
- Successfully demonstrated automatic identification of structural models that maximize likelihood for pharmacokinetic datasets.
Conclusions:
- Reinforcement learning offers a viable approach to automate the structural model identification in PopPK analysis.
- The developed framework can be extended to include more complex actions, such as covariate addition/removal or non-linear compartment integration.
Related Concept Videos
01:12Analysis of Population Pharmacokinetic Data
344
Analysis of population pharmacokinetic data involves studying the behavior of drugs within diverse populations to understand their pharmacokinetic parameters. Traditional pharmacokinetic methods typically involve collecting samples from a few individuals and estimating these parameters. While these methods are commonly used, they have limitations in capturing the variability in drug response among individuals or heterogeneous populations. Population pharmacokinetics is employed to address these...
344
00:59Model-Independent Approaches for Pharmacokinetic Data: Noncompartmental Analysis
111
Noncompartmental analyses offer an alternative method for describing drug pharmacokinetics without relying on a specific compartmental model. In this approach, the drug's pharmacokinetics are assumed to be linear, with the terminal phase log-linear. This assumption allows for simplified analysis and interpretation of the drug's behavior in the body.
One important characteristic of noncompartmental analyses is that drug exposure increases proportionally with increasing doses. This...
One important characteristic of noncompartmental analyses is that drug exposure increases proportionally with increasing doses. This...
111
01:06Model Approaches for Pharmacokinetic Data: Distributed Parameter Models
112
Pharmacokinetic models are mathematical constructs that represent and predict the time course of drug concentrations in the body, providing meaningful pharmacokinetic parameters. These models are categorized into compartment, physiological, and distributed parameter models.
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
112
01:19Nonlinear Pharmacokinetics: Overview
502
Nonlinear or dose-dependent pharmacokinetics is a phenomenon that occurs when the pharmacokinetic parameters of certain drugs deviate from linear pharmacokinetics at higher doses. These drugs do not follow the expected first-order kinetics, where the rate of drug elimination is directly proportional to the drug concentration. Instead, they exhibit a nonlinear relationship, which can be attributed to several factors.
Nonlinearity can arise due to the saturation of plasma protein-binding or...
Nonlinearity can arise due to the saturation of plasma protein-binding or...
502
01:26Pharmacokinetic Models: Comparison and Selection Criterion
123
Physiological and compartmental models are valuable tools used in studying biological systems. These models rely on differential equations to maintain mass balance within the system, ensuring an accurate representation of the dynamic processes at play.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
123
01:14Analysis Methods of Pharmacokinetic Data: Model and Model-Independent Approaches
204
Drug disposition in the body is a complex process and can be studied using two major approaches: the model and the model-independent approaches.
The model approach uses mathematical models to describe changes in drug concentration over time. Pharmacokinetic models help characterize drug behavior in patients, predict drug concentration in the body fluids, calculate optimum dosage regimens, and evaluate the risk of toxicity. However, ensuring that the model fits the experimental data accurately...
The model approach uses mathematical models to describe changes in drug concentration over time. Pharmacokinetic models help characterize drug behavior in patients, predict drug concentration in the body fluids, calculate optimum dosage regimens, and evaluate the risk of toxicity. However, ensuring that the model fits the experimental data accurately...
204