Developing a quality scoring system for epidemiological surveys of genetic disorders

A prototype database of published articles containing data on the frequency of human inherited disorders has been developed for use in clinical contexts, in medical research, for epidemiological studies, and in the planning of genetic services. It can be accessed at http://www.uwcm.ac.uk/uwcm/mg/fidd/. The information available in the literature comes from a wide range of publications, not all can be described primarily as epidemiological surveys. A preliminary assessment indicated that the quality of published articles was highly variable. Very few published articles describe quality-scoring systems for epidemiological surveys in general. We have developed a new scoring system to assess the quality of published articles on genetic disease frequency based on five main criteria and nine component scores. We carried out a first pilot study, to test its feasibility, usefulness and reproducibility. Eleven assessors scored six papers on the epidemiology of Huntington disease. As a result of these findings, we modified the scoring system to define each component more clearly. This was then evaluated in a second pilot study, which utilized six assessors to score 20 papers. The consensus between assessors remained relatively poor. We feel that while it may be possible to improve the scoring system further, the degree of improvement attainable may be limited by the poor standards of content and presentation of published surveys, which led to confusion and uncertainty by assessors. This pilot exercise has shown the need to raise the awareness of researchers and clinicians about the basic quality standards to be required of epidemiological genetic surveys.