Know your population and know your model: Using model-based regression and poststratification to generalize findings...

Area of Science:

• Psychological science
• Survey research
• Quantitative psychology

Background:

• Psychology research often involves complex interactions, posing challenges for generalizing findings from nonrepresentative samples.
• Estimating average treatment effects requires robust methods that account for population variability.

Purpose of the Study:

• To propose and demonstrate an extension of multilevel regression and poststratification (MRP) for psychological research.
• To address challenges in generalizability and replication within the psychological sciences.

Main Methods:

• Fitting a statistical model that incorporates interactions between treatment and background variables.
• Averaging model predictions over the population distribution of background variables, extending the multilevel regression and poststratification (MRP) framework.
• Applying the extended MRP method to estimate the norming distribution for the Big Five Personality Scale using open-source data.

Main Results:

• The proposed method effectively estimates population-level parameters from potentially nonrepresentative samples.
• Demonstrated application to the Big Five Personality Scale provides a robust norming distribution.
• Highlights the potential of integrating large open data sources with MRP for psychological research.

Conclusions:

• The extended MRP approach offers a powerful tool for enhancing the generalizability of psychological research findings.
• Combining large-scale open data with MRP can help mitigate current issues with replication and generalizability in psychology.
• This methodology supports more reliable inference and broader applicability of psychological research.