Quantifying selective reporting and the Proteus phenomenon for multiple datasets with similar bias

Area of Science:

• Biostatistics
• Genetics
• Neuroscience

Background:

• Meta-analyses synthesize research but are hindered by reporting biases.
• Statistically non-significant findings are often underreported compared to significant ones.
• Bias patterns can be complex, influenced by research timing and prior publications.

Purpose of the Study:

• To develop and apply an approach for analyzing large-scale published results datasets.
• To investigate publication bias patterns in Alzheimer's disease genetic marker studies.
• To capture and correct for complex, dynamic patterns of selective reporting.

Main Methods:

• Developed a novel approach to analyze large datasets of published research findings.
• Investigated a dataset of 1167 results from Alzheimer's disease case-control studies on 102 genetic markers.
• Quantified publication bias for initial studies versus replications and assessed the Proteus phenomenon.

Main Results:

• Initial studies showed substantial publication bias against non-significant findings (44% relative publication chance).
• Replications exhibited less bias, with non-significant results having an 84% relative publication chance.
• The Proteus phenomenon was observed, with non-significant studies opposing initial findings more likely published (73% relative chance).

Conclusions:

• Conventional meta-analysis methods may fail to capture complex, dynamic publication bias patterns.
• The developed approach offers robust analysis by accounting for various coexisting bias types.
• Findings highlight the need for advanced methods to ensure reliable conclusions from published research, particularly in molecular medicine and genetics.