Measuring the aggregated impact of research: Establishing criteria for coding Translational Science Benefits Model...

Area of Science:

• Translational Science
• Research Impact Assessment
• Health Services Research

Background:

• The Translational Science Benefits Model (TSBM) assesses research impact across clinical, community, economic, and policy domains.
• Standardized methods for verifying TSBM data are currently lacking, hindering quantitative analysis and aggregation.
• Accurate assessment of research impact is crucial for evidence-based decision-making and resource allocation.

Purpose of the Study:

• To establish content and face validity for criteria used in verifying qualitative TSBM data.
• To develop a foundation for a standardized TSBM coding tool.
• To enable quantitative aggregation and comparison of research impact data.

Main Methods:

• A modified Delphi process involving 11 topic experts.
• Two survey rounds with a moderated discussion to reach consensus on criteria.
• Consensus defined as 70% agreement among panelists.

Main Results:

• 197 validated criteria were developed across 30 TSBM benefits.
• Criteria fall into 9 categories: Content Relevant, Project Related, Who, Reach, What, How, Novel, Documented Evidence, and When.
• The process established content and face validity for the criteria.

Conclusions:

• The validated criteria provide a foundation for a TSBM coding tool.
• Standardization of TSBM data verification will facilitate data aggregation and analysis.
• This work supports the quantitative assessment and comparison of research impact across diverse contexts.