Related Experiment Video
Updated: Jun 11, 2025
10:26
Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
Published on: September 11, 2021
3.9K
Dealing with imperfect randomization: Inference for the highscope perry preschool program
James Heckman1, Rodrigo Pinto2, Azeem M Shaikh1
1Department of Economics, University of Chicago.
Journal of Econometrics
|October 7, 2024
Summary
This study addresses imperfect randomization in program evaluations by using a partial identification approach. It successfully identified significant program effects on multiple outcomes, including reduced criminal activity.
Area of Science:
- Econometrics
- Causal Inference
- Program Evaluation
Background:
- Imperfect randomization complicates program effect estimation.
- Reassignment of treatment status based on observed/unobserved characteristics poses challenges.
Purpose of the Study:
- To develop a method for making valid inferences on multiple outcomes with imperfect randomization.
- To control the familywise error rate in hypothesis testing for program effects.
Main Methods:
- Partial identification approach utilizing information on randomization imperfections.
- Development of a procedure for testing a family of null hypotheses.
- Application to the HighScope Perry Preschool program data.
Main Results:
- Demonstrated the possibility of nontrivial inferences despite imperfect randomization.
- Found statistically significant program effects on various outcomes.
- Identified effects on criminal activity for both males and females.
Conclusions:
- The proposed methodology effectively handles imperfect randomization and multiple outcomes.
- The HighScope Perry Preschool program showed significant positive impacts.
- Robust statistical inferences can be made even with imperfect experimental designs.
Keywords:
C31Exact InferenceExperimentsFamilywise Error RateI21Imperfect RandomizationJ13Multiple OutcomesMultiple TestingPermutation TestsPerry Preschool ProgramProgram EvaluationMore Related Videos
Related Concept Videos
02:01Group Design
8.9K
The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between...
8.9K
01:13Randomized Experiments
6.7K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
6.7K
01:24Testing a Claim about Population Proportion
3.3K
A complete procedure for testing a claim about a population proportion is provided here.
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
3.3K
02:07Blind Procedures
10.6K
Ideally, the people who observe and record the children’s behavior are unaware of who was assigned to the experimental or control group, in order to control for experimenter bias. Experimenter bias refers to the possibility that a researcher’s expectations might skew the results of the study. Remember, conducting an experiment requires a lot of planning, and the people involved in the research project have a vested interest in supporting their hypotheses. If the observers knew which...
10.6K
01:50Statistical Significance
20.1K
Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. A statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this...
20.1K
01:15Study Design in Statistics
7.9K
A study design is a set of techniques that allow a researcher to collect and analyze data from different variables defined for a specific research problem. Statistics is commonly for effective study design and more robust experiments,
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer more effective at growing roses than another? Is fatigue as dangerous to a driver as the influence of alcohol? Questions like these are answered using randomized experiments with proper...
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer more effective at growing roses than another? Is fatigue as dangerous to a driver as the influence of alcohol? Questions like these are answered using randomized experiments with proper...
7.9K