Related Experiment Video
Updated: Jul 14, 2025
06:18
The Collective Trust Game: An Online Group Adaptation of the Trust Game Based on the HoneyComb Paradigm
Published on: October 20, 2022
2.1K
In models we trust: preregistration, large samples, and replication may not suffice
Martin Spiess1, Pascal Jordan1
1Institute of Psychology, Department of Psychology and Human Movement Science, University of Hamburg, Hamburg, Germany.
Frontiers in Psychology
|October 9, 2023
Summary
Statistical models in psychological research must account for data generation processes, especially concerning sample selection and assumption validity. Ignoring these factors, even with large samples or preregistration, can lead to biased conclusions and flawed inferences.
Area of Science:
- Psychological research methodology
- Statistical modeling in social sciences
Background:
- Replicability in psychological research is debated, yet selection mechanisms and statistical model assumptions are often overlooked.
- These overlooked issues question whether statistical tools accurately reflect the data generation process.
Purpose of the Study:
- To investigate the impact of selection mechanisms and violated model assumptions on the validity of inferences in psychological research.
- To highlight the limitations of common strategies like preregistration and large samples in addressing these issues.
Main Methods:
- Analysis of statistical inference validity under selective sampling conditions.
- Discussion of consequences of violated assumptions in linear models and generalized linear mixed models (e.g., in item response theory).
Main Results:
- Inferences from selective samples are only valid under very specific conditions, contrary to common practice.
- Violated model assumptions in psychological research are frequently ignored, leading to potential biases.
- Standard strategies (preregistration, large samples, replication) do not resolve these fundamental problems.
Conclusions:
- Addressing selection mechanisms and model assumptions is crucial for valid psychological research.
- Strategies like model diagnostics, methods for selectivity compensation, and non-parametric estimation can help mitigate bias.
- A fundamental solution involves iterative theory development with statistically justified assumptions and empirical research on self-selection.
Related Concept Videos
01:23Mechanistic Models: Compartment Models in Individual and Population Analysis
62
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
62
02:20Longitudinal Research
12.0K
Sometimes we want to see how people change over time, as in studies of human development and lifespan. When we test the same group of individuals repeatedly over an extended period of time, we are conducting longitudinal research. Longitudinal research is a research design in which data-gathering is administered repeatedly over an extended period of time. For example, we may survey a group of individuals about their dietary habits at age 20, retest them a decade later at age 30, and then again...
12.0K
02:01Group Design
8.9K
The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between...
8.9K
02:16Surveys
14.8K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
14.8K
01:16Contaminants and Errors
103
Effective sample preparation is crucial for accurate and reliable laboratory analysis. During this process, two significant sources of error can arise: concentration bias from improper sample splitting and contamination caused by methods used to reduce particle size, such as grinding or homogenization. Identifying and minimizing these potential errors is crucial to ensuring the validity of the analysis.
Another key consideration is determining the appropriate number of samples required to...
Another key consideration is determining the appropriate number of samples required to...
103
02:30Naturalistic Observations
15.5K
If you want to understand how behavior occurs, one of the best ways to gain information is to simply observe the behavior in its natural context. However, people might change their behavior in unexpected ways if they know they are being observed. How do researchers obtain accurate information when people tend to hide their natural behavior? As an example, imagine that your professor asks everyone in your class to raise their hand if they always wash their hands after using the restroom. Chances...
15.5K