Benchmarking biochemical networks generated by large language models

Area of Science:

• Computational Biology
• Systems Biology
• Biochemistry
• Artificial Intelligence in Science

Background:

• Computational models of biochemical networks are crucial for understanding cell decisions.
• Manual curation of these networks from literature is time-consuming and limited by incomplete data.

Purpose of the Study:

• To evaluate the capability of general-purpose large language models (LLMs) in generating accurate biochemical network models.
• To assess LLM performance in both signaling and metabolic network reconstruction.

Main Methods:

• LLMs were prompted to generate signaling networks for cardiomyocyte hypertrophy, myofibroblast activation, and mechanosignaling.
• LLMs were used to generate the core metabolic network of Escherichia coli.
• Logic-based models were constructed from LLM-generated networks to predict responses to perturbations and substrate utilization.

Main Results:

• LLMs generated 24-65% of reactions for literature-curated signaling networks.
• Logic-based models from LLM-generated signaling networks achieved 6-33% accuracy in predicting responses to perturbations.
• LLMs generated 64-91% of reactions for the E. coli metabolic network, with variable accuracy in substrate utilization prediction.

Conclusions:

• Current general-purpose LLMs can generate biochemical networks with moderate accuracy.
• This study establishes a pipeline and benchmarks for improving LLM-based biochemical network generation.
• LLMs show promise as a tool to accelerate the construction of computational models in systems biology.