Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Reliability and Validity01:29

Reliability and Validity

Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
Test for Homogeneity01:23

Test for Homogeneity

The goodness–of–fit test can be used to decide whether a population fits a given distribution, but it will not suffice to decide whether two populations follow the same unknown distribution. A different test, called the test for homogeneity, can be used to conclude whether two populations have the same distribution. To calculate the test statistic for a test for homogeneity, follow the same procedure as with the test of independence. The hypotheses for the test for homogeneity can be stated as...
Group Design02:01

Group Design

The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between the two are due to...
Testing a Claim about Standard Deviation01:19

Testing a Claim about Standard Deviation

A complete procedure to test a claim about population standard deviation or population variance is explained here.
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
Comparing Experimental Results: Student's t-Test01:09

Comparing Experimental Results: Student's t-Test

The t-test is a statistical method used to compare the sample mean with a population mean or compare two means from two data sets. The test statistic is calculated from the standard deviation, mean, and number of measurements in the data set at a selected confidence interval and then compared to a table of critical values at this confidence level. If the test statistic is smaller than the critical value, the null hypothesis is accepted. In this case, we state that the difference between the...
Behrens–Fisher Test00:57

Behrens–Fisher Test

The Behrens-Fisher test is a statistical method designed to address the Behrens-Fisher problem, which arises when comparing the means of two normally distributed populations with unequal variances. Unlike the Student's t-test, which assumes equal variances, the Behrens-Fisher test allows for mean comparison without this restrictive assumption. This flexibility makes it particularly valuable in scenarios where two independent samples exhibit normality but lack variance homogeneity.
This test is...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

When Do Unifactorial Items Increase the Reliability?

Psychometrika·2026
Same author

Bias and precision in true-score estimation.

The British journal of mathematical and statistical psychology·2026
Same author

Recognize the Value of the Sum Score, Psychometrics' Greatest Accomplishment.

Psychometrika·2026
Same author

Proof of Reliability Convergence to 1 at Rate of Spearman-Brown Formula for Random Test Forms and Irrespective of Item Pool Dimensionality.

Psychometrika·2026
Same author

Reliability Theory for Measurements with Variable Test Length, Illustrated with ERN and Pe Collected in the Flanker Task.

Psychometrika·2026
Same author

Rejoinder to McNeish and Mislevy: What Does Psychological Measurement Require?

Psychometrika·2026
Same journal

The Rhythm Reproduction Task for children: A psychometric examination.

Behavior research methods·2026
Same journal

QualGames: A Qualtrics implementation and a database of behavioral game theory tasks.

Behavior research methods·2026
Same journal

PyLossless: A non-destructive EEG processing pipeline.

Behavior research methods·2026
Same journal

"It makes sense to me": Examining data file column names in the visual cognitive literature.

Behavior research methods·2026
Same journal

Planned missingness in intensive longitudinal studies: Extensions and comparisons of multiform designs.

Behavior research methods·2026
Same journal

A validity-guided workflow for robust large language model research in psychology.

Behavior research methods·2026
See all related articles

Related Experiment Video

Updated: May 21, 2026

A Protocol of Manual Tests to Measure Sensation and Pain in Humans
07:28

A Protocol of Manual Tests to Measure Sensation and Pain in Humans

Published on: December 19, 2016

A standard for test reliability in group research.

Jules L Ellis1

  • 1School of Psychology and Artificial Intelligence, Radboud University Nijmegen, P.O. Box 9104, 6500 HE, Nijmegen, The Netherlands. j.ellis@psych.ru.nl

Behavior Research Methods
|June 28, 2012
PubMed
Summary
This summary is machine-generated.

Researchers propose a new standard for evaluating test reliability in experiments. This standard balances test costs with statistical power, aiming to maximize experimental efficiency and reduce overall research expenses.

More Related Videos

The Collective Trust Game: An Online Group Adaptation of the Trust Game Based on the HoneyComb Paradigm
06:18

The Collective Trust Game: An Online Group Adaptation of the Trust Game Based on the HoneyComb Paradigm

Published on: October 20, 2022

Related Experiment Videos

Last Updated: May 21, 2026

A Protocol of Manual Tests to Measure Sensation and Pain in Humans
07:28

A Protocol of Manual Tests to Measure Sensation and Pain in Humans

Published on: December 19, 2016

The Collective Trust Game: An Online Group Adaptation of the Trust Game Based on the HoneyComb Paradigm
06:18

The Collective Trust Game: An Online Group Adaptation of the Trust Game Based on the HoneyComb Paradigm

Published on: October 20, 2022

Area of Science:

  • Psychometrics
  • Experimental Psychology
  • Research Methodology

Background:

  • Traditional guidelines suggest test reliabilities of .70 or .80 for group research.
  • Existing standards do not account for the economic factors of experimental design.
  • Optimizing reliability is crucial for maximizing statistical power within resource constraints.

Purpose of the Study:

  • Introduce a novel standard for evaluating test reliability based on experimental costs.
  • Demonstrate how this standard can enhance statistical power while minimizing research expenses.
  • Propose a new benchmark for the reliability of dependent variables in experimental research.

Main Methods:

  • Developed a cost-based framework for determining optimal test reliability.
  • Calculated efficient reliability values based on the proportion of test administration costs to total experimental costs.
  • Illustrated the relationship between efficient reliability and maximized statistical power.

Main Results:

  • An example shows that if test administration costs are 7% of total costs, the efficient reliability is .93.
  • Achieving efficient reliability maximizes statistical power for a given budget.
  • Aligning actual reliability with efficient reliability enhances experimental outcomes.

Conclusions:

  • The proposed standard offers a more nuanced approach to test reliability in experimental research.
  • Adopting this standard can lead to more powerful and cost-effective experiments.
  • This framework provides a practical guideline for researchers to optimize their study designs.