Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Reliability and Validity01:29

Reliability and Validity

Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
Factorial Design02:01

Factorial Design

Factorial Analysis is an experimental design that applies Analysis of Variance (ANOVA) statistical procedures to examine a change in a dependent variable due to more than one independent variable, also known as factors. Changes in worker productivity can be reasoned, for example, to be influenced by salary and other conditions, such as skill level. One way to test this hypothesis is by categorizing salary into three levels (low, moderate, and high) and skills sets into two levels (entry level...
Friedman Two-way Analysis of Variance by Ranks01:21

Friedman Two-way Analysis of Variance by Ranks

Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures from...
Theory of Attribution II: Kelley's Covariation Theory01:29

Theory of Attribution II: Kelley's Covariation Theory

Attribution theory plays a crucial role in social psychology, helping to explain how individuals interpret the causes of behavior. One prominent model within this field is Harold Kelley's covariation theory, which provides a systematic approach to determining whether internal traits or external circumstances drive a person's actions. The model posits that individuals rely on three key types of information—consensus, consistency, and distinctiveness—to make these judgments.Consensus: Comparing...
Confirmation Biases01:31

Confirmation Biases

The confirmation bias is the tendency to focus on information that confirms our existing beliefs and ignore information that is inconsistent with our expectations. For example, if you think that your professor is not very nice, you notice all of the instances of rude behavior exhibited by the professor while ignoring the countless pleasant interactions he is involved in on a daily basis. Have you ever fallen prey to the confirmation bias, either as the source or target of such bias?
Confidence Coefficient01:24

Confidence Coefficient

The confidence coefficient is also known as the confidence level or degree of confidence. It is the percent expression for the probability, 1-α, that the confidence interval contains the true population parameter assuming that the confidence interval is obtained after sufficient unbiased sampling; for example, if the CL = 90%, then in 90 out of 100 samples the interval estimate will enclose the true population parameter. Here α is the area under the curve, distributed equally under both the...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

New methodological and software tools for probing moderation in intrinsically nonlinear models.

Behavior research methods·2026
Same author

Association of community sociodemographic and tuberculosis-related factors with variability in community-level tuberculosis stigma in South Africa: an ecological analysis from the MISSED TB Outcomes study.

BMJ open·2026
Same author

Single-Level Bifactor Models as Implicit Multilevel Factor Models Without a Bifactor Structure.

Multivariate behavioral research·2026
Same author

Development of a New Measure of Intergenerational Contact.

Journal of adult development·2025
Same author

Determining the power of a 1-sided z-test given only the power of the corresponding 2-sided test.

Journal of behavioral medicine·2025
Same author

Community variability in TB-related stigma in South Africa: an ecologic analysis from the MISSED TB Outcomes Study.

medRxiv : the preprint server for health sciences·2025
Same journal

Bayesian evaluation for latent variable models: A tutorial on computing information criteria and bayes factors with the r package bleval.

Psychological methods·2026
Same journal

A stochastic block prior for clustering in graphical models.

Psychological methods·2026
Same journal

Three-level vector autoregressive models.

Psychological methods·2026
Same journal

Scaling cognitive modeling to big data: A deep learning approach to studying individual differences in evidence accumulation model parameters.

Psychological methods·2026
Same journal

Best practices in multilevel modeling for within-cluster group comparisons: An evaluation of coding strategies reflecting group composition and heterogeneity.

Psychological methods·2026
Same journal

A unified framework for psychometrics in experimental psychology: The standardized generalized hierarchical factor model.

Psychological methods·2026
See all related articles

Related Experiment Video

Updated: May 11, 2026

Applying an eMASS Customization Program as a Research Tool to Evaluate Consumer Benefits
08:27

Applying an eMASS Customization Program as a Research Tool to Evaluate Consumer Benefits

Published on: September 27, 2019

Reliability estimation in a multilevel confirmatory factor analysis framework.

G John Geldhof1, Kristopher J Preacher2, Michael J Zyphur3

  • 1Institute for Applied Research in Youth Development, Tufts University.

Psychological Methods
|May 8, 2013
PubMed
Summary
This summary is machine-generated.

Assessing measurement reliability in multilevel sampling requires examining reliability at each specific level. Standard single-level estimates can be misleading, but two-level reliability measures like alpha and omega generally perform well.

More Related Videos

Advancing Dyslexia Assessment in Children Through Computerized Testing
09:00

Advancing Dyslexia Assessment in Children Through Computerized Testing

Published on: August 16, 2024

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
10:39

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning

Published on: August 29, 2025

Related Experiment Videos

Last Updated: May 11, 2026

Applying an eMASS Customization Program as a Research Tool to Evaluate Consumer Benefits
08:27

Applying an eMASS Customization Program as a Research Tool to Evaluate Consumer Benefits

Published on: September 27, 2019

Advancing Dyslexia Assessment in Children Through Computerized Testing
09:00

Advancing Dyslexia Assessment in Children Through Computerized Testing

Published on: August 16, 2024

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
10:39

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning

Published on: August 29, 2025

Area of Science:

  • Psychometrics
  • Multilevel Modeling
  • Statistical Methods

Background:

  • Measurement reliability is crucial in multistage sampling with nested data structures.
  • Existing methods often fail to account for reliability at multiple levels of analysis (e.g., individual, group).
  • Lack of clear guidance complicates accurate reliability assessment in complex sampling designs.

Purpose of the Study:

  • To highlight the importance of assessing level-specific reliability in multilevel contexts.
  • To introduce and evaluate methods for estimating multilevel reliability.
  • To provide practical guidance and code for researchers.

Main Methods:

  • Utilized multilevel confirmatory factor analysis (MCFA) for reliability estimation.
  • Conducted a simulation study to compare different reliability metrics.
  • Included an applied example with Mplus and R code for reproducibility.

Main Results:

  • Single-level reliability estimates are inaccurate unless reliability is uniform across all levels.
  • Two-level alpha and composite reliability (omega) demonstrated good performance.
  • Maximal reliability (H) estimates showed significant bias in multilevel data.
  • Small cluster sizes can inflate between-level reliability estimates.

Conclusions:

  • Level-specific reliability assessment is essential for accurate measurement evaluation in multilevel studies.
  • Two-level reliability coefficients (alpha, omega) are recommended for practical application.
  • Researchers should be cautious of potential biases, especially with small cluster sizes and certain reliability metrics.