Monitoring Open Science as transformative change: Towards a systemic framework

Following a flurry of policies for Open Science (OS), there is now a wave of initiatives to monitor its adoption. However, the great diversity of understandings and activities related to Open Science makes monitoring very challenging. There is a danger that by focusing on what can be readily observed (e.g. publications) many other OS activities are overlooked (e.g. participation), with a potential narrowing of OS scope, streetlight effects, and deviation from the values of OS. Since Open Science can be understood as a systemic transformation of the research system, we have borrowed concepts from Transformative Innovation Policies frameworks which aim at evaluating socio-technical transitions. In accordance with this view of OS as a systemic transformation, we propose that the new monitoring efforts should shift towards: (i) systemic perspectives which considers the various actions related to OS, including policies and outputs (e.g. datasets) but also processes (e.g. participatory events), outcomes (e.g. citizen interest in science) and expected impacts (e.g. better scientific contributions to addressing societal problems); (ii) implementation of monitoring as reflexive learning (rather than accountability or benchmarking); (iii) mapping the directionality of the activities and the values associated with the choices in directions. In summary, a monitoring framework for OS requires a profound change in conventional monitoring practices. The scope should broaden from current focus on outputs (such as publications) towards the processes of connection that make science 'open' (usage, co-creation and dialogue), as well as towards outcomes (changes in practices) and the longer-term impacts that reflect the values and normative commitments of OS.