Intercept studies, clinical trials, and cluster experiments: to whom can we extrapolate?

Area of Science:

• Epidemiology
• Biostatistics
• Health Services Research

Background:

• Traditional sample surveys assume a clear link between sample and population.
• Non-survey designs, like intercept studies and clinical trials, often lack this explicit relationship.
• Inferences from these designs to the broader population are frequently based on unstated assumptions about observation probabilities.

Purpose of the Study:

• To compare intercept studies and clinical trials regarding population inference.
• To emphasize the challenges in generalizing findings when probabilities of observation are unknown, unreliable, or very small.
• To highlight the need for robust methodologies in non-survey research designs for valid population-based conclusions.

Main Methods:

• Comparative analysis of intercept studies (e.g., mall intercepts, migrant population studies) and clinical trials.
• Examination of implicit and explicit assumptions regarding population representation in these study designs.
• Focus on the probabilities of observing population members in different research contexts.

Main Results:

• Intercept studies require assumptions about the probability of encountering population members.
• Clinical trials often exclude large population segments, resulting in zero probability of observation for them.
• Both designs face challenges in applying results broadly when observation probabilities are poorly understood.

Conclusions:

• There is a critical need to address the unknown or unreliable probabilities of observation in non-survey research.
• Methodological advancements are required to strengthen the link between study samples and the target populations for valid inference.
• Future research should focus on developing strategies to improve the generalizability of findings from intercept studies and clinical trials.