Toward improved evidence standards and methods for rehabilitation: recommendations and challenges

Interventions and programs for people with disability should be based on the best--the most discriminating and rigorous--methods of systematic review and knowledge translation possible. Extant systems for systematic review and practice recommendations have excellent features but severe difficulties are encountered when attempting to apply them to disability and rehabilitation. This article identifies issues in evidence synthesis and linked practice recommendations and describes both new and long-tested methods to address them. Evidence synthesis in disability and rehabilitation can be improved by: explicating criteria for evaluating nonrandomized evidence, including the regression discontinuity, interrupted time series, and single-subject designs, as well as state-of-the-art methods of analysis of observational studies; greater use of meta-analysis; considering effect size, direction of biases, and dose-response relationships; employing more discriminating methods of evaluating flaws in masking, considering also measurement reliability and objectivity; considering overall biases and conflicts of interest; increased attention to composition of review panels; and greater transparency in reporting of the bases of reviewers' judgments. Review methods need to be developed for assistive technology and for measurement procedures. Application to practice can be improved by attention to treatment alternatives, explicit evaluation of generalizability, synthesizing clinical experience as a source of evidence, and a focus on the best--rather than the ideally most-rigorous--evidence. Study outcomes should be measured and reviewed in terms meaningful to persons served. In sum, methods are available to improve evidence synthesis and the application of resulting knowledge. We recommend that these methods be employed.