On the importance and validity of medical accreditation standards

In late 1997, the authors conducted a national survey of communities of interest about the importance and clarity of 44 accreditation standards applied to teaching, learning, and evaluation in medical schools by the Liaison Committee on Medical Education (LCME). Questionnaires were mailed to deans and educational administrators at U.S. medical schools; current LCME members and surveyors and those who had served during the preceding five years; a random selection of residency program directors drawn from both general practice and speciality disciplines; sample groups of medical students and residents; and a cohort of practicing physicians not affiliated with academic medical institutions. Altogether 1,659 questionnaires were mailed, and 701 responses were received (42%). The recipients were asked to use a five-point Likert scale to rate each of the 44 standards both for its perceived importance as an indicator of the quality of undergraduate medical education and for the clarity with which the standard's intent was conveyed. Although the mean ratings of importance all fell in the "moderately important" and "highly important" areas across the respondent groups, the ratings divided into three groups, semantically and statistically. At the high end for importance are standards dealing with fundamental qualities of students, instruction, and the structuring of resources. At the low end of the importance scale are standards dealing largely with matters of process. The ratings for clarity were systematically lower than the ratings for importance, and in some cases the rating for clarity were even more widely discrepant with the ratings for importance. Individual comments by respondents about certain standards were critical of their complicated construction and of confusion about their meaning and measures of compliance. One or more of these hallmarks--being rated of lower importance or clarity, and being the target of criticism by survey respondents--distinguished most of the standards that earlier study had shown are often neglected by surveyors. The predictive validity of each of a number of standards was examined by testing the association between the standard (or its derivative) and outcomes expressed in annual student and school questionnaires and compiled in databases of the Association of American Medical Colleges and the American Medical Association. The result was a mixed bag, confounded by the absence of specific dimensions of many accreditation standards (independent variables) and the lack of discriminating measures of outcome (dependent variables). Nevertheless, the LCME's accreditation standards are believed to be important by those most affected by them. And beyond validating that medical accreditation is guided by relevant standards for teaching, learning, and evaluation, the results of this study point to ways by which the process can be made more precise and useful.