Related Experiment Video
Updated: May 8, 2026
10:26
Problem-Solving Before Instruction (PS-I): A Protocol for Assessment and Intervention in Students with Different Abilities
Published on: September 11, 2021
US studies may overestimate effect sizes in softer research.
Daniele Fanelli1, John P A Ioannidis
1Science, Technology and Innovation Studies, The University of Edinburgh, Edinburgh EH1 1LZ, United Kingdom.
Summary
Scientific research biases are worsened by low consensus and publication bias. US-based researchers in behavioral studies may show a propensity for stronger findings, unlike non-US studies.
Area of Science:
- Health research
- Biological and behavioral sciences
Background:
- Scientific research is affected by numerous biases, leading to resource waste, health risks, and hindered progress.
- These issues may be exacerbated by a lack of consensus in theories and methods, selective publication, and overly productive career systems, particularly in the United States (US).
Purpose of the Study:
- To investigate how primary study outcomes deviate from meta-analysis summary effect sizes.
- To identify potential differences in research biases between US-based and non-US-based researchers, particularly in behavioral versus nonbehavioral studies.
Main Methods:
- Extracted 1,174 primary outcomes from 82 meta-analyses in Genetics & Heredity and Psychiatry.
- Measured the deviation of individual study results from the overall effect size within each meta-analysis.
- Analyzed outcomes based on the inclusion of behavioral parameters and the geographical location of the corresponding author.
Main Results:
- Primary studies with behavioral outcomes were more likely to report extreme effects.
- US-based corresponding authors were more prone to deviate towards hypothesized effects, especially in non-biological behavioral studies.
- Nonbehavioral studies showed no "US effect"; sampling variance and small-study effects were more prominent, particularly for non-US countries.
Conclusions:
- The observed "US effect" in behavioral research may stem from lower methodological consensus and higher noise, potentially leading US researchers to report stronger findings.
- Publication bias against non-US authors cannot be ruled out for nonbehavioral studies, but the US effect in behavioral research is less likely due to editorial biases.
Keywords:
publish or perishquestionable research practicesresearch biasscientific misconductsoft scienceMore Related Videos
Related Concept Videos
02:16Surveys
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
01:52Regression Toward the Mean
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when researchers try to extrapolate results...
01:37Statistical Significance
Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. A statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this...
01:22Bias
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
01:53Cause and Effect
While variables are sometimes correlated because one does cause the other, it could also be that some other factor, a confounding variable, is actually causing the systematic movement in our variables of interest. For instance, as sales in ice cream increase, so does the overall rate of crime. Is it possible that indulging in your favorite flavor of ice cream could send you on a crime spree? Or, after committing crime do you think you might decide to treat yourself to a cone?
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...