Mapping variation in dual use risk assessments of synthetic biology projects

When a life science project is identified as potential dual use research of concern (DURC), United States government policy and biorisk management professionals recommend conducting a risk assessment of the project and using its results to choose strategies to manage any associated risks. However, there is little empirical research on how real-world projects score on DURC assessments, the extent to which reviewers agree or disagree about risks for a given project, or how risk judgments map to recommended risk management strategies. By studying the process of DURC risk assessment, it may be possible to develop methods that are more consistent, accurate, and cost-effective.

Using a modified version of the framework in the Companion Guide to the United States Government Policies for Oversight of Life Sciences Dual Use Research of Concern, we elicited detailed reviews from 18 experienced DURC reviewers and 49 synthetic biology students of the risks, benefits, and recommended risk management strategies for four real-world synthetic biology projects.

We found significant variation among experts, as well as between experts and students, in both perceived DURC risk and recommended risk management strategies. For some projects, expert risk assessments spanned 4 out of 5 possible ratings. We found substantial disagreement between participants about the appropriate actions to take to manage the DURC risks of each project.

The observed variation in participants' judgments suggests that decisions for similar projects may vary significantly across institutions, exposing the public to inconsistent standards of risk management. We provide several research-based suggestions to reduce reviewer disagreement and manage risk more efficiently when reviewers disagree.