Related Experiment Video
Updated: May 31, 2026
10:26
Problem-Solving Before Instruction (PS-I): A Protocol for Assessment and Intervention in Students with Different Abilities
Published on: September 11, 2021
Risk factors and interventions with statistically significant tiny effects
George C M Siontis1, John P A Ioannidis
1Clinical Trials and Evidence-Based Medicine Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.
International Journal of Epidemiology
|July 9, 2011
Summary
Statistically significant tiny effects are increasingly common in medical literature. However, these small effect sizes may be due to bias, warranting cautious interpretation of their clinical or public health importance.
Area of Science:
- Epidemiology
- Biostatistics
- Medical Research
Background:
- Large-scale studies can detect statistically significant findings with minimal effect sizes.
- Investigating the interpretation of these small, yet significant, relative risks (RRs) is crucial.
Purpose of the Study:
- To empirically assess a large number of statistically significant relative risks (RRs) of small magnitude.
- To evaluate how investigators interpret these tiny effects.
Main Methods:
- Identified RRs between 0.95 and 1.05 in cohort study abstracts from top medical journals and Cochrane reviews.
- Recorded study design, participant data, risk factors/interventions, outcomes, effect estimates, P-values, and author interpretations.
- Calculated the probability of effects falling outside specific intervals around the null value.
Main Results:
- Evaluated 51 tiny effects, predominantly in nutrition, biomarkers, and healthcare correlates.
- Most effects (37/51) were published between 2006-2010.
- While 15 effects had >80% probability of being outside the 0.97-1.03 RR interval, none were highly likely to be outside the 0.90-1.10 interval.
- Concerns about small magnitude, confounding, or bias were discussed for 23 effects, but not for 28.
Conclusions:
- Statistically significant, small effects are increasingly prevalent in clinical and public health research.
- Cautious interpretation is essential, as minimal biases can eliminate these effects, leaving their true importance uncertain.
Related Concept Videos
01:52Regression Toward the Mean
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when researchers try to extrapolate results...
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
01:37Statistical Significance
Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. A statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this...
01:25Statistical Methods for Analyzing Epidemiological Data
Epidemiological data primarily involves information on specific populations' occurrence, distribution, and determinants of health and diseases. This data is crucial for understanding disease patterns and impacts, aiding public health decision-making and disease prevention strategies. The analysis of epidemiological data employs various statistical methods to interpret health-related data effectively. Here are some commonly used methods:
01:12What is an Experiment?
An experiment is a planned activity carried out under controlled conditions. The purpose of an experiment is to investigate the relationship between two variables. When one variable causes change in another, we call the first variable the explanatory or independent variable. The affected variable is called the response or dependent variable. In a randomized experiment, the researcher manipulates values of the explanatory variable and measures the resulting changes in the response variable. The...
01:29Bias in Epidemiological Studies
Biases can arise at various stages of research, from study design and data collection to analysis and interpretation. Recognizing and addressing these biases is essential to ensure the validity and reliability of epidemiological findings.Broadly speaking, biases in epidemiology fall into three main categories: selection bias, information bias, and confounding. A more detailed description of possible biases is: