[Psychotherapy research--how it should (not) be done. An expert reanalysis of comparative studies by Grawe et al....

22 psychotherapy outcome studies used in the Bernese meta-analysis by Grawe, Donati and Bernauer (1994) for a treatment comparison between behavioural and psychoanalytic-psychodynamic treatment concepts were reanalysed by 12 expert psychotherapy researchers independent of each other. Three clinical criteria served as basic criteria for the assessment of the methodological quality of the 22 comparative studies: treatment dosage, therapists' competence or expertise, and an adequate realisation of the intended treatment concept. The expert ratings were then compared with an evaluation of an assessment of a research team from the University of Ulm and with the one from the Bernese research team. Contrary to the Bernese meta-analysis, both the experts and the Ulm research group conclude that only 5 or 8 of the 22 studies, respectively, could be accepted for a relatively fair comparison between the treatments under study. Out of the 5/8 studies, none could be considered fully suitable for a treatment comparison, at the most only moderately suitable. These remaining 5 or 8 studies, respectively, do not prove superiority of one treatment over the other. Hence, the "Dodo Bird Verdict" stands up under scrutiny.