Evaluating a CTSA-funded pilot grant program

Pilot programs are integral to catalyzing and accelerating research at Clinical and Translational Science Award (CTSA) hubs. However, little has been published about the structure and operationalization of pilot programs or how they impact the translational research enterprise at CTSAs. The North Carolina Translational and Clinical Science Institute (NC TraCS), the CTSA hub at the University of North Carolina at Chapel Hill (UNC-CH) conducted an evaluation case study to describe the pilot program structure, assess process outcomes, and provide a framework for other institutions to utilize for the evaluation of their respective pilot programs.

We describe the operationalization of our pilot program, the evaluation framework utilized to evaluate the program, and how we analyzed available data to understand how our pilot funding opportunities were utilized by investigators. We calculated application volumes and funding rates by investigator position title and pilot application type. We also reviewed feedback provided by pilot Principal Investigators (PIs) to understand how many pilot projects were completed, NC TraCS service utilization, and barriers to research. Limited data on publications and subsequent funding was also reviewed.

Between 2009 and 2019 the NC TraCS Pilot Program received 2343 applications and funded 933 projects, ranging from $2000 to $100,000 in amount, with an overall funding rate of 39.8%. Utilization of NC TraCS services had positive impacts on both resubmission funding and project completion rates.

This process evaluation indicates that the program is being operationalized in a way that successfully fulfills the program mission while meeting the needs of a diverse group of researchers.