Enhancing review criteria for dissemination and implementation science grants

Area of Science:

• Dissemination and Implementation Science (DIS)
• Health Services Research
• Grant Review Methodologies

Background:

• Existing grant review criteria inadequately address the unique methods and priorities of Dissemination and Implementation Science (DIS).
• The ImplemeNtation and Improvement Science Proposals Evaluation CriTeria (INSPECT) was developed to address this gap, comprising 10 criteria based on established DIS principles.
• This study describes the adaptation and application of the INSPECT scoring system alongside the NIH scoring system for evaluating pilot DIS research proposals.

Purpose of the Study:

• To adapt and evaluate the INSPECT scoring system for assessing pilot Dissemination and Implementation Science (DIS) grant proposals.
• To compare the utility of the adapted INSPECT criteria with the standard NIH scoring system in the context of DIS research.
• To identify the strengths and limitations of both scoring systems for evaluating DIS research.

Main Methods:

• Adapted the INSPECT criteria to encompass diverse DIS settings and concepts, including explicit inclusion of dissemination and implementation methods.
• Trained five PhD-level researchers with DIS expertise to review seven grant applications using both INSPECT and NIH criteria.
• Conducted independent reviews by two researchers per grant, followed by group discussions and a follow-up survey to gather reviewer feedback on the scoring process.

Main Results:

• Averaged scores indicated a range of 13–24 for INSPECT and 2–5 for NIH criteria.
• Reviewers found INSPECT valuable for assessing the integration of DIS considerations, generalizability, feasibility, and impact.
• NIH criteria were better suited for effectiveness-focused or pre-implementation proposals, while INSPECT facilitated evaluation of DIS-specific elements.

Conclusions:

• Using both INSPECT and NIH criteria demonstrated complementarity in evaluating pilot DIS grant proposals.
• INSPECT serves as a valuable resource for DIS training and capacity building, aiding in proposal development.
• Potential refinements for INSPECT include enhanced guidance for pre-implementation proposals and clearer rating criteria.