GPT-4 as a biomedical simulator.

Area of Science:

• Computational Biology
• Biomedical Informatics
• Artificial Intelligence in Medicine

Background:

• Computational simulation accelerates biomedical research but requires significant expertise.
• Large language models (LLMs) demonstrate broad task applicability.
• This study explores LLMs as biological system simulators.

Purpose of the Study:

• To demonstrate the proof-of-concept for using GPT-4 as a versatile biological system simulator.
• To introduce SimulateGPT, a knowledge-driven simulation approach.
• To evaluate SimulateGPT's performance against direct LLM inference.

Main Methods:

• SimulateGPT utilizes structured prompting of GPT-4 for knowledge-driven simulations.
• Evaluations included four qualitative scenarios (e.g., mouse experiments, sepsis treatment) and two quantitative scenarios (gene essentiality, survival prediction).
• Benchmarking involved blinded qualitative assessments by domain experts and quantitative analysis against experimental data.

Main Results:

• Biomedical scientists preferred SimulateGPT's predictions over direct GPT-4 inference in qualitative evaluations.
• SimulateGPT significantly improved accuracy in predicting gene essentiality.
• The approach enhanced correlation and precision for predicting patient progression-free survival.

Conclusions:

• LLMs, particularly text-based simulators like SimulateGPT, offer a novel approach for modeling complex biological systems.
• This method is suitable for systems with extensive textual knowledge, complementing physics-based simulations.
• Future directions include integrating web search, mathematical modeling, and experimental data fine-tuning.