Balancing precision with inclusivity in meta-analyses: A response to Roos and colleagues (2017)

Roos and colleagues' commentary on our recent meta-analysis examining acute stress effects on executive functions is an important delineation of the limits of meta-analyses with heterogeneous outcomes. In this response, we wish to both clarify the inferences we feel are appropriate given the analyses and address the methodological concerns they raise. Additionally, we present new analyses that answer questions raised in their commentary. We first discuss the classification of a broad array of tasks that depend upon the same construct (e.g., inhibition) and note that this allows for inferences regarding the process that underpins all of those tasks, but this does not entail that all tasks dependent upon that construct will evidence the same effect of stress. Second, we argue that requiring that a study present a significant effect of stress on cortisol for inclusion in analyses is too stringent for a number of reasons (e.g., some studies using validated stressor paradigms correctly do not assay cortisol for budgetary reasons) and we present analyses showing that even when studies that did not present a cortisol response were removed, the initially observed effects still held. Finally, we address concerns raised regarding analyses of time-dependent effects by presenting new analyses that help to allay those concerns. In sum, we applaud Roos and colleagues' exortation for greater methodological and conceptual rigor in studies of stress and executive function, and the additional analyses prompted by their questions help to clarify observed effects and further the field of stress and executive function research.