Comparing Methods to Denote Treatment Outcome in Clinical Research and Benchmarking Mental Health Care

Approaches based on continuous indicators (the size of the pre-to-post-test change; effect size or ΔT) and on categorical indicators (Percentage Improvement and the Jacobson-Truax approach to Clinical Significance) are evaluated to determine which has the best methodological and statistical characteristics, and optimal performance, in comparing outcomes of treatment providers. Performance is compared in two datasets from providers using the Brief Symptom Inventory or the Outcome Questionnaire. Concordance of methods and their suitability to rank providers is assessed. Outcome indicators tend to converge and lead to a similar ranking of institutes within each dataset. Statistically and conceptually, continuous outcome indicators are superior to categorical outcomes as change scores have more statistical power and allow for a ranking of providers at first glance. However, the Jacobson-Truax approach can complement the change score approach as it presents outcome information in a clinically meaningful manner. Copyright © 2015 John Wiley & Sons, Ltd.

When comparing various indicators or treatment outcome, statistical considerations designate continuous outcomes, such as the effect size of the pre-post change (effect size or ΔT) as the optimal choice. Expressing outcome in proportions of recovered, changed, unchanged or deteriorated patients has supplementary value, as it is more easily interpreted and appreciated by clinicians, managerial staff and, last but not the least, by patients. If categorical outcomes are used with small datasets, true differences in institutional performance may get obscured due to diminished power to detect differences. With sufficient data, outcome according to continuous and categorical indicators converge and lead to similar rankings of institutes' performance.