Steps toward developing an algorithm to facilitate recognition of translational science

In preparing for our Clinical and Translational Science Award (CTSA) UM1 application, we recognized the need to develop a shared understanding of the distinctions between translational science (TS) and translational research (TR). We describe our efforts to develop and evaluate the reliability of a concise instrument that investigators and reviewers could use to distinguish between TS and TR.

Groups of faculty and staff individually reviewed published translational studies to determine whether the project involved TS and, separately, TR. One group (n = 10) first reviewed 14 publications with limited guidance; the same group and a second group (n = 9) then reviewed another set of 14 publications guided by a detailed algorithm. We used kappa statistics to measure agreement in the determinations of TS and TR for each publication.

The overall kappa coefficients in the three sets of TS determinations (two by the first group and one by the second group) were 0.61, 0.33, and 0.18, respectively. The overall kappa coefficients in the three sets of TR determinations were 0.26, 0.11, and 0.40, respectively. The median kappa coefficients for all 42 determinations were 0.39 for TS and 0.22 for TR, both indicating only fair agreement. We found no evidence that the algorithm helped to improve agreement rates.

Our results show gaps in understanding the distinction between TS and TR among CTSA hub faculty and staff. We discuss some reasons for this gap and propose ways that could improve the recognition of TS and TR.