[Critical comments on the statistical methods in Grawe, Donati and Bernauer: "Change in psychotherapy. From...

In the meta-analysis "Changing Psychotherapy" by Grawe, Donati and Bernauer different psychotherapeutic methods are compared based upon published therapy studies. Hereby the authors claim also to have proven with statistical methods that certain kinds of therapy are more effective than others. I show here that the descriptive and inductive methods used are not able to withstand a critical examination; they are incorrect and in most cases even inadmissible. The results of my examination show that there are four points of critique: 1. The question of how effective a kind of therapy is, according to Grawe's criteria, depends more on the number of variables and their measurements with which a therapy is judged than on the number of patients examined in the single studies. 2. Grawe does not distinguish between dependent and independent variables or measurements; every measurement of each variable is included in his methods with the same weight. 3. The different effect variables used to evaluate the therapic process are mostly represented on varying ordinal scales which are incomparable with each other. Grawe treats these scales as if they were comparable, often even as if they were metric. 4. All five statistical methods (counting significances, binomial test, profile of difference values, t-test, Wilcoxon-test) with which Grawe evaluates the results of the single studies are inadmissible because the conditions required are not met. In sum: The conclusions stated in the meta-analysis cannot be seen as being statistically validated or statistically proven.