A decision making approach to assessing critical appraisal skills

Critical appraisal is an important skill for practicing evidence-based medicine. Most assessments of critical appraisal focus either on knowledge about research design or evaluate a written critique of an article. This paper presents a Bayesian framework for assessing critical appraisal through measuring how clinical decisions and judgments change with the introduction of new research evidence. In a skilled clinician-appraiser, new evidence is predicted to cause the clinician to revise beliefs in the appropriateness of a decision, in proportion to both the strength of the findings and the methodological soundness of the research which produced the findings. The framework is illustrated by two studies using different items, respondents, and designs.