Reliability of adverse drug reaction assessment in psychiatric inpatients

Within an ongoing drug surveillance program in psychiatric hospitals the applicability of an algorithm for judgment on probability of causal relationship of adverse events and drug therapy was tested. Algorithmic interrater agreement was compared to agreement obtained with the conventional criteria used so far within the program in 80 cases by two raters, who had participated in the drug surveillance program since its beginning in 1979. With the use of the algorithm raters agreed on imputed drugs in 86% of all cases; total agreement on drugs and degree of probability was obtained in 69% (weighted kappa 0.618). Raters agreed on total score for the imputed medication in 49% and also on all subscores for the different axes of the algorithm in 43% of all cases. Differentiation of drug-related from illness-related changes, the use of judgmental terms within the algorithm and specific problems created by the frequent use of combinations of drugs with similar profiles of adverse drug reactions (ADR) in psychiatric patients were identified as the main sources of disagreement. Agreement on total judgment was comparable to results from similar studies in the literature using various algorithms, but in contrast to all these studies a higher percent of agreement (80%) was obtained with the use of the conventional criteria in this study.