Representation Meets Optimization: Training PINNs and PIKANs for Gray-Box Discovery in Systems Pharmacology

Area of Science:

• Computational Science
• Biomedical Engineering
• Machine Learning

Background:

• Physics-Informed Neural Networks (PINNs) and Physics-Informed Kolmogorov-Arnold Networks (PIKANs) are advanced models for inverse problems and system identification.
• A comprehensive understanding of PIKANs' and PINNs' performance regarding accuracy and speed is lacking.
• Existing research has not fully explored the impact of optimizers and architectural choices on these models.

Purpose of the Study:

• To systematically investigate the influence of optimizers, representations, and training configurations on the performance of PINNs and PIKANs.
• To introduce and evaluate a modified PIKAN architecture, tanh-cPIKAN, for enhanced performance.
• To provide practical guidance for selecting optimal models and training strategies in systems pharmacology.

Main Methods:

• Benchmarking a wide array of optimizers (first-order, second-order, hybrid) and learning rate schedulers using the Optax library.
• Evaluating the impact of model architecture (MLP vs. KAN), numerical precision (single vs. double), and warm-up phases.
• Assessing optimizer scalability for larger models and analyzing JAX-related trade-offs in computational efficiency and numerical accuracy.
• Utilizing two systems pharmacology case studies: a pharmacokinetics model and a chemotherapy drug-response model.

Main Results:

• Identified effective optimizer and representation combinations for learning gray-box models under challenging conditions (ill-posed, non-unique, data-sparse).
• Demonstrated the performance variations influenced by model architecture, numerical precision, and initial learning rates.
• Quantified the trade-offs associated with JAX implementation regarding computational speed and accuracy.
• The tanh-cPIKAN architecture showed promise for enhanced performance.

Conclusions:

• The study offers practical insights into selecting appropriate optimizers and architectures for robust and efficient gray-box discovery in systems pharmacology.
• Findings provide actionable guidance for improving the training of physics-informed networks in biomedical applications.
• Optimizing training configurations is crucial for maximizing the accuracy and speed of PIKANs and PINNs.