Failure to account for psychiatric symptoms: Implications for the replicability and generalizability of psychological...

One of the challenges of psychological research is obtaining a sample representative of the general population. One largely overlooked participant characteristic is sub-clinical levels of psychiatric symptoms.

A series of studies were conducted to assess (i) whether typical psychology study participants had more psychiatric symptoms than the general population, (ii) whether there are sub-groups defined by psychiatric symptoms within the no-diagnosis, no-medication participant pool, and (iii) whether sub-clinical levels of psychiatric symptoms have an effect on standard behavioral tasks. Five UK national datasets (N > 10,000) were compared to data from psychology study participants (Study 1: n = 872; Study 2: n = 43,094; Study 3: n = 267).

Psychology study participants showed significantly higher levels of anxiety and depression and lower well-being, according to four commonly used mental health measures (GHQ-12, PHQ-8, WEMWBS, and WHO-5). Five sub-groups within the psychology study participant group were identified based on symptom levels, ranging from none to significant psychiatric symptoms. These groupings predicted performance on tests of executive function, including the Stroop task and the n-back task, as well as measures of intelligence.

This study demonstrates that standard psychology participant pools are unrepresentative and suggests that a failure to account for psychiatric symptoms when recruiting for any psychological study is likely to negatively impact the reproducibility and generalizability of psychological science.