"Insufficient evidence of effectiveness" is not "evidence of no effectiveness:" evaluating computer-based education...

This article reports on commissioned research funded by the Swedish Council of Technology Assessment in Health Care (SBU) and the Swedish Nursing Society (SSF). The objective was to review computer-based education programs. However, as the review produced insufficient evidence of effectiveness, the publication was withheld due to a previous incident where such evidence was misunderstood by Swedish policy and health care decision makers.

This article highlights the concept of evidence with regard to the consequences of insufficient evidence of effectiveness being mistaken for evidence of no effectiveness. The aim is also to present a systematic review evaluating a computer-based education program for patients suffering from severe mental illness.

Systematic database searches in Medline, CINAHL, PsycINFO and the Cochrane Library identified a total of 131 potentially relevant references. Thereafter, 27 references were retrieved as full-text documents, of which 5 were finally included and co-reviewed by two independent researchers.

The review found no decisive evidence of effectiveness regarding computer-based education programs designed to assist persons suffering from severe mental illness.

Failing to see the difference between insufficient evidence and evidence of no effectiveness may have unexpected consequences. As a result, practice may be misguided and treatments withheld, which at worse may have harmful consequences for patients. In the end, it is of utmost importance that researchers do good quality research by ensuring statistical power and quality of outcome measurement. For example, this review of computer-based education programs could have revealed effective ways of dealing with severe mental illness if the studies included had been conducted using more sophisticated designs.