Assessing methods for causality assessment of suspected adverse drug reactions

Reproducibility and validity as currently defined are inappropriate criteria for the evaluation of methods for causality assessment. Reproducibility leads to suppression rather than resolution of real disagreements and the method used to establish validity relies on the tarnished gold standard of expert opinion. We describe six alternative criteria that attempt to address a potential user's main concerns--the need to know whether to believe the results in general and in a particular case. These criteria focus on the internal structure of a method rather than its output. When we assessed the published methods by these criteria most of the methods failed most of the criteria. We believe that the problem and its solution lie at a fundamental level--real understanding of the true nature of causality assessment, which we suggest is an inherently subjective evaluation based on the multiple uncertainties that an assessor has about a case and not an objective attribute of the drug-event connection that can be determined from unambiguous evidence elicited in response to "operational questions".