Producing credible evaluations of federal health programs

Based on extensive experience as a reviewer of evaluations of federal health programs, the author distinguishes and discusses two sources of evidence for the credibility of an evaluation. Substantive credibility is indicated by the quality of the literature review; the evaluative understanding manifest in the description of the program or treatment; the relation between the study questions and the study design; the expertise and candor displayed in the methods discussion; and the degree to which stated findings and recommendations are supported by methods and data. Presentational credibility is undermined by language that reveals subjectivity or advocacy, but it is supported by the competent organization and readability of the report.