Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Reliability and Validity01:29

Reliability and Validity

Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs01:15

Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs

Bioequivalence experimental study designs play a pivotal role in testing the effectiveness of various treatments. Key among these are the repeated measures, cross-over, carry-over, and Latin square designs. In the repeated measures design, each subject receives all treatments, allowing for temporal comparisons. This type of design is useful in reducing variability but requires careful planning to avoid bias.The cross-over design, an economical method, involves sequential administration of...
Naturalistic Observations02:30

Naturalistic Observations

If you want to understand how behavior occurs, one of the best ways to gain information is to simply observe the behavior in its natural context. However, people might change their behavior in unexpected ways if they know they are being observed. How do researchers obtain accurate information when people tend to hide their natural behavior? As an example, imagine that your professor asks everyone in your class to raise their hand if they always wash their hands after using the restroom. Chances...
Randomized Experiments01:13

Randomized Experiments

The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Group Design02:01

Group Design

The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between the two are due to...
Random and Systematic Errors01:20

Random and Systematic Errors

Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Identification of hub genes involved in early-onset schizophrenia: from genetic susceptibility to predicted regulated gene expression.

Molecular psychiatry·2026
Same author

Familial coaggregation and shared genetic loading of mental disorders and cardiovascular diseases.

Psychological medicine·2026
Same author

Transparency of and preferences for Picture Communication Symbols and VoiceSymbol among young Taiwanese children with typical development.

Augmentative and alternative communication (Baltimore, Md. : 1985)·2026
Same author

Cryo-EM structure of the SRI-HtrI complex reveals the cytoplasmic coupling in an archaeal phototaxis system.

Nature communications·2026
Same author

Glibenclamide attenuates neuronal pyroptosis by modulating K<sub>ATP</sub> channels in the hippocampus of palmitic acid-induced lipotoxic mice.

Biomedical journal·2026
Same author

Tamoxifen Use and Risk of Uterine Diseases in Young Women With Breast Cancer.

Obstetrics and gynecology·2026

Related Experiment Video

Updated: Jun 4, 2026

Use of a Video Scoring Anchor for Rapid Serial Assessment of Social Communication in Toddlers
09:16

Use of a Video Scoring Anchor for Rapid Serial Assessment of Social Communication in Toddlers

Published on: March 14, 2018

Bayesian random effects for interrater and test-retest reliability with nested clinical observations.

Chuhsing K Hsiao1, Pei-Chun Chen, Wen-Hsin Kao

  • 1Department of Public Health and Institute of Epidemiology and Preventive Medicine, College of Public Health, National Taiwan University, Taipei, Taiwan. ckhsiao@ntu.edu.tw

Journal of Clinical Epidemiology
|February 5, 2011
PubMed
Summary

This study introduces a Bayesian hierarchical correlation model to jointly assess inter- and intrarater reliability, even with complex data structures and dichotomous responses. The model provides a flexible framework for evaluating reproducibility across multiple raters and time points.

More Related Videos

Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity
08:40

Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity

Published on: June 12, 2019

Related Experiment Videos

Last Updated: Jun 4, 2026

Use of a Video Scoring Anchor for Rapid Serial Assessment of Social Communication in Toddlers
09:16

Use of a Video Scoring Anchor for Rapid Serial Assessment of Social Communication in Toddlers

Published on: March 14, 2018

Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity
08:40

Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity

Published on: June 12, 2019

Area of Science:

  • Statistics
  • Biostatistics
  • Medical Informatics

Background:

  • Assessing inter- and intrarater reliability often involves complex nested data structures and multiple raters.
  • Existing methods struggle to jointly model both inter- and intrarater reliability, especially with dichotomous outcomes.
  • Nesting structures, covariates, and multiple dependence sources complicate reliability inference.

Purpose of the Study:

  • To develop a unified statistical framework for simultaneously assessing inter- and intrarater reliability.
  • To address challenges in modeling reliability with dichotomous responses and complex data structures.
  • To provide a flexible and feasible approach for reproducibility analysis.

Main Methods:

  • Established equivalence between correlation and kappa for multiple raters.
  • Applied a Bayesian generalized linear mixed-effects model.
  • Incorporated correlated random effects for raters and time to infer similarity and test-retest reliability, accounting for covariates and nesting.

Main Results:

  • The proposed Bayesian hierarchical correlation model accommodates individual covariates, and random effects for subjects, raters, and time.
  • The model demonstrates applicability to diverse data structures and types.
  • Illustrated with an application in endodontic radiographic examinations.

Conclusions:

  • The Bayesian hierarchical correlation model offers a versatile, adaptable, and practical solution for joint inter- and intrarater reliability modeling.
  • This approach enhances the ability to evaluate reproducibility in complex research settings.