Controlling familywise error rates and false discovery rates when studying predictors of log-normal durations of...

Studies in anaesthesiology frequently use generalised linear models to identify quantitatively important 'independent predictors' of log-normally distributed outcomes, such as surgical and anaesthesia times. However, the performance of common multiple-comparison procedures at preventing type I and II errors is unknown for these problems.

We conducted Monte Carlo simulations to evaluate methods for controlling the familywise error rate (FWER) and false discovery rate (FDR). Simulated datasets had log-normal outcomes and three binary predictors, with varying correlation among them (independent, strong positive, or moderate negative). We applied four FWER (Bonferroni, Šidák, Holm-Bonferroni, and Hochberg) and two FDR (Benjamini-Hochberg and Benjamini-Yekutieli) procedures to the P-values derived from the generalised linear models.

Without adjustment for multiple comparisons, the FWER was large (12.6-14.8% instead of the correct [nominal] 5.0%). Among FWER methods, the Bonferroni adjustment was the most accurate, with rates consistently close to the nominal 5.0% level across all correlation scenarios (5.2-5.3%). For FDR control, the Benjamini-Yekutieli procedure was effective for independent and negatively correlated predictors (4.5-5.1%) but failed to control the FDR under strong positive predictor correlation (6.0-9.5%).

When using generalised linear models to identify predictors of log-normal outcomes, the simplest approach, Bonferroni adjustment, provided reliable control of the FWER. The Benjamini-Yekutieli procedure is the most suitable for controlling the FDR, but our findings show it can be anti-conservative (i.e. unreliable) when potential predictors of the anaesthesia times are positively correlated (i.e. precisely the conditions that would generally hold for these problems).