Dental Students' Use of AMSTAR to Critically Appraise Systematic Reviews

The idea of basing clinical procedures upon evidence gathered by observation is less than 200 years old, with the first set of evidence-based position papers dating back only to the early 1970s. The relationship between evidence-based education and health outcomes is difficult to test and may be indirect, but teaching critical appraisal skills may be beneficial in developing knowledge. Systematic reviews have a central role in the process of clinical decision making in practice and therefore should be of high quality, following a rigorous protocol that can be evaluated with validated tools. The aim of this study was to assess how dental students utilized the Assessment of Multiple Systematic Reviews (AMSTAR) appraisal tool to evaluate systematic reviews in the context of a treatment planning course. During the in-class final exam, students were required to appraise the quality of a systematic review and to justify their answers. Of the 74 third-year students who took the exam, 100% answered all questions on the AMSTAR form. The mean number of correct answers was nine (SD=1.047, Min=6, Max=10), with no student providing all 11 correct answers. The fact that nearly 90% of the students provided eight or more correct answers suggests that AMSTAR can be used by students to evaluate the methodological quality of systematic reviews. It also was evident that although the AMSTAR tool requires less than 15 minutes to complete an evaluation, using it requires extensive training and repetition to achieve consistent and reliable results.