Balance of power: The choice between trial and participant numbers to optimise the detection of phase-dependent effects

The fields of neuroscience and psychology are currently in the midst of aso-called reproducibility crisis, with growing concerns regarding a history ofweak effect sizes and low statistical power in much of the research published inthese fields over the last few decades. Whilst the traditional approach foraddressing this criticism has been to increaseparticipantsample sizes, there are many research contexts in which the number oftrialsper participant may be of equal importance. Thepresent study aimed to compare the relative importance of participants andtrials in the detection of phase-dependent phenomena, which are measured acrossa range of neuroscientific contexts (e.g., neural oscillations, non-invasivebrain stimulation). This was achievable within a simulated environment where onecan manipulate the strength of this phase dependency in two types of outcomevariables: one with normally distributed residuals (idealistic) and onecomparable with motor-evoked potentials (an MEP-like variable). We compared thestatistical power across thousands of experiments with the same number ofsessions per experiment but with different proportions of participants andnumber of sessions per participant (30 participants × 1 session, 15participants × 2 sessions, and 10 participants × 3 sessions), withthe trials being pooled across sessions for each participant. These simulationswere performed for both outcome variables (idealistic and MEP-like) and fourdifferent effect sizes (0.075-"weak,"0.1-"moderate," 0.125-"strong,"0.15-"very strong"), as well as separate control scenarioswith no true effect. Across all scenarios with (true) discoverable effects, andfor both outcome types, there was a statistical benefit for experimentsmaximising the number of trials rather than the number of participants (i.e., itwas always beneficial to recruit fewer participants but have them complete moretrials). These findings emphasise the importance of obtaining sufficientindividual-level data rather than simply increasing number of participants.