Randomized controlled versus naturalistic studies: a new research agenda

The present article addresses the question of what kind of evidence is required to demonstrate that a method of psychotherapy works. Referring to recent conceptualizations of the logical structure of scientific theories, that is, the structuralistic view of theories, the author shows that randomized controlled studies (RCTs) and naturalistic studies (effectiveness studies) refer to different domains of intended applications (laboratory vs. field). This view has several important implications: (1) RCTs and naturalistic studies do not differ concerning their internal and external validity; (2) naturalistic studies do not necessarily provide lower-level evidence than RCTs; (3) evidence from RCTs cannot be transferred to psychotherapeutic practice in the field; (4) naturalistic studies are required to demonstrate that a form of therapy works in the field; (5) The proposed catalogues for levels of evidence focus on RCTs; thus, they cannot be applied to the question if a therapy works in the field; (6) It is necessary to define separate criteria for levels of evidence of naturalistic studies; and (7) a new research agenda for naturalistic studies can be derived, which is analogous to that of efficacy studies. In this article, a proposal is made to define levels of evidence of naturalistic studies. A gold standard for naturalistic studies is proposed.