Dual-use artificial intelligence and biology: upstream risk-benefit reviews

Biological AI models (BAIMs) are advancing rapidly and hold substantial promise. Yet, these models raise dual-use concerns, particularly regarding capabilities that could enhance pathogens with pandemic potential. Current risk mitigation discussions for BAIMs are concentrated in the post-development stage, focusing on evaluations and safeguards, after a model has been trained. We argue that upstream, pre-development risk-benefit review (RBR) is a necessary, missing component of effective BAIM governance. The broad range of models, their capabilities, and purpose-built applications, as well as a majority-academic developer community, make this approach feasible and realistic. We propose a review framework with the following five components and discuss key characteristics of each component: (1) review trigger criteria that determine whether a model should undergo an RBR. The review trigger criteria are (A) a model is reasonably anticipated to possess capabilities of concern or (B) it will be trained on sensitive pathogen data classified under the Biosecurity Data Levels (BDL) system; structured risk (2) and benefit (3) reviews using qualitative and quantitative criteria; (4) integration of risk and benefit scores into a composite assessment; and (5) proportionate risk mitigation recommendations. We expect that such reviews (2 through 5) would apply to only a small fraction of BAIMs and would benefit responsible developers by establishing clear expectations at the outset of model development. We discuss how such a framework could be implemented broadly across academic institutions, commercial developers, and federal, philanthropic, and private funding bodies, and address limitations such as the subjectivity of risk assessments. RBR for BAIMs remains nascent and will require expert-driven working groups to define capabilities of concern, establish clear review criteria, and assess risk mitigation efficacy. RBRs are a promising conceptual approach for BAIM risk management and should be pioneered, refined, and vetted through real-world application with model developers.