Monitoring Open Science as transformative change: Towards a systemic framework

Area of Science:

• Science and Technology Policy
• Research Evaluation
• Societal Impact of Science

Background:

• Numerous Open Science (OS) policies have been enacted, leading to initiatives for monitoring adoption.
• The diverse interpretations and activities within OS present significant challenges for effective monitoring.
• Current monitoring often overemphasizes observable outputs like publications, potentially neglecting other crucial OS aspects such as participation.

Purpose of the Study:

• To propose a shift in monitoring practices for Open Science (OS).
• To address the limitations of current monitoring approaches that may narrow the scope of OS.
• To align monitoring with the concept of OS as a systemic transformation of the research system.

Main Methods:

• Borrowing concepts from Transformative Innovation Policies frameworks for evaluating socio-technical transitions.
• Advocating for systemic perspectives that encompass policies, outputs, processes, outcomes, and impacts.
• Recommending monitoring as a reflexive learning process rather than for accountability or benchmarking.
• Emphasizing the mapping of activity directionality and associated values.

Main Results:

• Current monitoring practices require profound change.
• Monitoring scope should expand beyond outputs (e.g., publications) to include processes (e.g., co-creation, dialogue).
• Monitoring should also encompass outcomes (e.g., changes in practices) and long-term impacts aligned with OS values.

Conclusions:

• A comprehensive OS monitoring framework necessitates broadening the focus from outputs to processes, outcomes, and impacts.
• Monitoring should adopt a systemic view, integrating diverse OS activities and their contributions to societal problems.
• The proposed framework supports reflexive learning and ensures alignment with the normative commitments of Open Science.