Related Experiment Video
Updated: Jul 4, 2026
09:49
Holistic Facial Composite Creation and Subsequent Video Line-up Eyewitness Identification Paradigm
Published on: December 24, 2015
Can jurors recognize missing control groups, confounds, and experimenter bias in psychological science?
Bradley D McAuliff1, Margaret Bull Kovera, Gabriel Nunez
1Department of Psychology, California State University, Northridge, CA 91330-8255, USA. bradley.mcauliff@csun.edu
Law and Human Behavior
|July 1, 2008
Summary
Jury members could not reliably detect internal validity threats in psychological science evidence. Only the absence of a control group significantly impacted perceptions of evidence quality and expert credibility.
Area of Science:
- Psychological Science
- Legal Psychology
- Scientific Reasoning
Background:
- Juries evaluate scientific evidence in legal trials.
- Understanding how mock jurors assess the internal validity of psychological studies is crucial for legal proceedings.
Purpose of the Study:
- To assess jury-eligible individuals' ability to identify internal validity threats in psychological research presented in a trial context.
- To examine the impact of internal and ecological validity on perceptions of expert evidence and credibility.
Main Methods:
- Participants (N=248) reviewed case summaries featuring expert testimony on psychological studies with manipulated internal validity (valid, missing control, confound, experimenter bias) and ecological validity (high, low).
- Evaluated were ratings of expert evidence quality, expert credibility, and plaintiff credibility, alongside verdict decisions.
Main Results:
- Higher ratings for expert evidence quality and credibility were observed only when comparing valid studies to those missing a control group.
- Internal validity threats did not influence verdict decisions or ratings of plaintiff credibility.
- Ecological validity did not significantly affect any outcome measures.
Conclusions:
- Mock jurors demonstrated limited ability to detect nuanced internal validity threats beyond the absence of a control group.
- Perceptions of expert evidence and credibility were influenced by basic study design flaws, but not more complex threats.
- Verdict decisions were positively correlated with perceived expert evidence quality, expert credibility, and plaintiff credibility, highlighting the importance of these factors regardless of internal validity detection.
Related Concept Videos
01:13Controls in Experiments
When conducting an experiment, it is crucial to have control to reduce bias and accurately measure the dependent variables. It also marks the results more reliable. Controls are elements in an experiment that have the same characteristics as the treatment groups but are not affected by the independent variable. By sorting these data into control and experimental conditions, the relationship between the dependent and independent variables can be drawn. A randomized experiment always includes a...
02:07Blind Procedures
Ideally, the people who observe and record the children’s behavior are unaware of who was assigned to the experimental or control group, in order to control for experimenter bias. Experimenter bias refers to the possibility that a researcher’s expectations might skew the results of the study. Remember, conducting an experiment requires a lot of planning, and the people involved in the research project have a vested interest in supporting their hypotheses. If the observers knew which child was...
01:31Confirmation Biases
The confirmation bias is the tendency to focus on information that confirms our existing beliefs and ignore information that is inconsistent with our expectations. For example, if you think that your professor is not very nice, you notice all of the instances of rude behavior exhibited by the professor while ignoring the countless pleasant interactions he is involved in on a daily basis. Have you ever fallen prey to the confirmation bias, either as the source or target of such bias?
01:11Blinding
Blinding is a commonly used method of not telling participants which treatment a subject is receiving. Blinding is a critical part of a randomized control trial or RCT. It reduces the bias that affects the results. In an RCT, blinding is used in the form of a placebo. A placebo effect occurs when untreated subjects falsely believe they have received the treatment and report improved symptoms. A placebo or a dummy treatment is administered to subjects to negate the bias caused by such an effect.
01:37Statistical Significance
Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. A statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this...
01:22Bias
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...