Principles and performance: assessing the evidence

Despite the apparent rigour of the systematic review process there are a number of reasons why it may not accurately reflect the true state of current knowledge. For inborn errors of intermediary metabolism in general, the formal quality of literature on treatment is poor, partly because the rarity of the conditions makes systematic study difficult, partly because of the self-evident evidence paradox with treatments that are so obviously life-saving, and partly because of the widespread use of biochemical normalisation as a proxy outcome. In the field of newborn screening there is a paucity of formal trials so that assessment must be either by the 'causal pathway' approach or by reference to screening "criteria". The causal pathway approach may be misleading in that there are discontinuities, for example in projecting from known incidence (clinically diagnosed cases) to the incidence expected on screening. The criteria-based approach suffers from qualitative descriptors, such as "important" and "adequate", with no clear end-points, whilst in the context of neonatal screening some of the Wilson and Jungner criteria conflict with current evidence-based views. These problems are illustrated by reference to two reviews recently produced as part of the UK Health Technology Assessment Programme.