Identifying spin in health management evaluations

Decision-makers in payor and provider organizations rely on the peer-reviewed literature as a source of ideas for new interventions to control costs and improve the quality of health care. However, recent evidence has emerged that spin tactics are regularly employed in the description of results from randomized controlled trials (RCTs) that actually do not show statistically significant outcomes. Observational studies, which are more commonly used to evaluate health management interventions than RCTs, offer greater opportunity for spin because evaluators control more aspects of the evaluation approach. This paper provides a detailed description of how to critically review study outcomes from health management interventions and identify spin. Our emphasis on health management is motivated by the tremendous discrepancy between the large financial savings reported by commercial vendors and the savings reported in RCTs and systematic reviews that indicate little-to-no financial benefit. We use unpublished data from a medical home pilot project to demonstrate how easily statistically non-significant findings can be portrayed in a favourable light, either through error, omission or intentional spin. We then describe additional techniques that should be utilized in order to present outcomes in an accurate and comprehensive manner. The step-by-step approach described here will hopefully assist readers in becoming more critical consumers of outcomes reported in scholarly journals or the popular media by identifying when spin tactics are used to camouflage ineffective interventions.