Assessing the significance of treatment effects: comments from the perspective of ethics

The process of designing research protocols for testing treatment effects and reporting the results of such research demands both explicit and implicit value judgments. When measuring the effects of alternative treatments, those judgments can include decisions about what outcomes to measure and for which persons they should be measured, how to aggregate disparate outcomes, and how to decide whether differences are substantial enough to differentiate between treatments. Researchers and reviewers often do not acknowledge problems of this sort, and default solutions are based on common usage rather than any epistemologic justification. Acknowledging the value commitments inherent in choices about research methods should serve to increase variation in methods rather than constrain them within conventional patterns. Especially with research intended to shape policy and practice, more thoughtful choices, humility in recognizing the limits of available methods, and flexibility in establishing the thresholds for significant differences for each study seem to be justified. The choices researchers make should be documented and the reasons for those choices should be given explicitly in publications and presentations so that readers and other users of the information are enabled and expected to bear more responsibility for interpreting and applying the findings appropriately.