Pilot testing of the Research Impact Assessment Framework.

Area of Science:

• Health services research
• Research evaluation
• Knowledge translation

Background:

• Traditional academic impact metrics often fail to capture the real-world value of research.
• There is a need for frameworks that assess research alignment with health system priorities.
• The Research Impact Assessment Framework was developed to address these limitations.

Purpose of the Study:

• To pilot the Research Impact Assessment Framework for feasibility and utility.
• To assess the framework's ability to evaluate research impact aligned with health, social, and economic priorities.
• To compare institutional research impact capacity using combined domain outputs.

Main Methods:

• Piloting the framework with health and research organizations in Sydney, Australia.
• Assessing the Research Environment domain via an employee survey.
• Evaluating the Alignment and Influence of Research domain through case study reviews.
• Combining domain outputs using synthetic data for institutional comparison.

Main Results:

• The framework's assessment tools are feasible for evaluating an organization's capacity for impactful research.
• Case study reviews highlighted challenges in distinguishing potential versus realized research impact.
• Implementation can be facilitated by using routinely collected data and artificial intelligence tools.

Conclusions:

• The Research Impact Assessment Framework is a feasible tool for assessing research impact aligned with health priorities.
• Utilizing existing data sources and AI can streamline implementation and improve consistency.
• Further refinement is needed to effectively measure realized research impact.