Claims of sex differences: an empirical assessment in genetic associations

Many studies try to probe for differences in risks between men and women, and this is a major challenge in the expanding literature of associations between genetic variants and common diseases or traits.

To evaluate whether prominently claimed sex differences for genetic effects have sufficient internal and external validity.

We searched PubMed through July 6, 2007, for genetic association studies claiming sex-related differences in the articles' titles. Titles and abstracts and, if necessary, the full text of the article were assessed for eligibility.

Two hundred fifteen articles were retrieved by the search. We considered eligible all retrieved association studies that claimed different genetic effects across sexes of 1 or more gene variants for any human disease or phenotype. We considered both biallelic and multiallelic markers (including haplotypes) and both binary and continuous phenotypes and traits. We excluded non-English-language studies; studies evaluating only 1 sex; studies in which sex was treated only as an independent predictor of disease; studies that did not address any association of the investigated genetic variant with a disease or trait; studies not involving humans; and studies in which the authors did not claim any sex difference.

Two evaluators independently extracted data with a third evaluator arbitrating their discrepancies. Data evaluation included whether analyses were stated to have been specified a priori; whether sex effects were evaluated in the whole study or subgroups thereof; and whether the claims were appropriately documented, insufficiently documented, or spurious. For appropriately and insufficiently documented claims we performed the calculations for gene-sex interaction whenever raw data were available. Finally, we compared the sex-difference claims with the best internal validity against the results of other studies addressing the same interaction.

We appraised 432 sex-difference claims in 77 eligible articles. Authors stated that sex comparisons were decided a priori for 286 claims (66.2%), while the entire sample size was used in 210 (48.6%) claims. Appropriate documentation of gene-sex interaction was recorded in 55 claims (12.7%); documentation was insufficient for 303 claims and spurious for the other 74. Data for reanalysis of claims were available for 188 comparisons. Of these, 83 (44.1%) were nominally statistically significant at a P = .05 threshold, and more than half of them (n = 44) had modest P values between .01 and .05. Of 60 claims with seemingly the best internal validity, only 1 was consistently replicated in at least 2 other studies.

In this sample of highly prominent claims of sex-related differences in genetic associations, most claims were insufficiently documented or spurious, and claims with documented good internal and external validity were uncommon.