Related Experiment Videos
Implications of direct and indirect range restriction for meta-analysis methods and findings
John E Hunter1, Frank L Schmidt, Huy Le
1Department of Psychology, Michigan State University, MI, USA.
The Journal of Applied Psychology
|June 2, 2006
Summary
Most meta-analysis methods incorrectly correct for indirect range restriction, leading to underestimated correlations. New methods accurately correct for indirect range restriction, revealing potentially significant underestimations in previous studies.
Area of Science:
- Psychometrics
- Industrial-Organizational Psychology
- Meta-analysis Methodology
Background:
- Range restriction is common in psychological and performance data.
- Existing meta-analysis techniques often fail to adequately correct for indirect range restriction.
- This leads to systematic underestimation of true score correlations.
Purpose of the Study:
- To identify and correct for the undercorrection bias in meta-analyses due to indirect range restriction.
- To develop accurate meta-analysis methods for data with indirect range restriction.
- To re-evaluate the correlation between general mental ability and job performance.
Main Methods:
- Distinguishing between direct and indirect range restriction.
- Developing novel statistical corrections for indirect range restriction in meta-analysis.
- Applying the new methods to a large-sample empirical database.
Main Results:
- Current meta-analysis methods result in substantial undercorrections when range restriction is indirect.
- The newly developed methods provide accurate corrections for indirect range restriction.
- Previous meta-analyses likely underestimated the general mental ability-job performance correlation by approximately 25%.
Conclusions:
- Failure to account for indirect range restriction is a significant methodological flaw in meta-analysis.
- Accurate corrections are crucial for validly estimating effect sizes, particularly the general mental ability-job performance relationship.
- This methodological issue has broad implications for meta-analytic research across various fields.
Related Concept Videos
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
01:29Pharmacodynamic Models: Direct Effect Model and Indirect Response Model
Pharmacodynamic models are essential tools in understanding the relationship between drug concentrations and their effects on biological systems. By characterizing the dynamics of drug action, these models guide dose selection, optimize therapeutic efficacy, and inform the development of new drugs. Two major classes of pharmacodynamic models include direct effect and indirect response models.Direct Effect ModelsDirect effect models describe the immediate relationship between drug concentration...
01:52Regression Toward the Mean
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when researchers try to extrapolate results...
01:21Friedman Two-way Analysis of Variance by Ranks
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures from...
01:23Mechanistic Models: Compartment Models in Individual and Population Analysis
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least squares (OLS)...
01:40Archival Research
Some researchers gain access to large amounts of data without interacting with a single research participant. Instead, they use existing records to answer various research questions. This type of research approach is known as archival research. Archival research relies on looking at past records or data sets to look for interesting patterns or relationships. For example, a researcher might access the academic records of all individuals who enrolled in college within the past ten years and...