Automating Knowledge-Driven Model Recommendation: Methodology, Evaluation, and Key Challenges

Computational modeling is important for understanding biological systems, however models of biological signaling networks are often constructed manually, limiting their size and complexity. Machine reading can extract molecular interactions of cellular networks described in the scientific literature and collected in databases to provide extensive knowledge that could be incorporated automatically into executable models if reliable methods for the assembly, extension and evaluation of models were available. Here, we evaluated the utility of breadth-first addition (BFA) and depth-first addition (DFA) algorithms when assembling or extending models with different structures: 100 randomly generated networks of Erdös-Rényi and Barabási-Albert types, and two published executable models of intracellular signaling. We simulated the assembled Candidate Models with stochastic simulator DiSH and computed their steady-state total model error (TME). The BFA and DFA methods led to maximum 65% recall, indicating that information is still missing in assembled models even when their TME is low. We examined the impact that target network structure, available baseline model information, and evaluation method have on the ability to automatically assemble and extend synthetic and published models using the breadth-first or depth-first approaches. Although achieving desired TME values, our results highlight that these methods produce simplified cellular signaling models, suggesting that more sophisticated methods are needed for the assembly, extension and evaluation of complex, dynamic, biological networks.