Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Reliability and Validity01:29

Reliability and Validity

13.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
13.7K
Statistical Analysis: Overview01:11

Statistical Analysis: Overview

14.0K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
14.0K
Kendall's Coefficient of Concordance01:20

Kendall's Coefficient of Concordance

915
Kendall's Coefficient of Concordance (W), also known as Kendall's W, is a non-parametric statistical measure used to assess the agreement or concordance between multiple raters or judges when they rank a set of items. It is often used when you have ordinal data (ranks) and you want to see if there is consistency or consensus among the raters. It is widely applied in research areas such as psychology, medicine, and social sciences, where multiple judges are asked to rank or rate subjects...
915
Group Design02:01

Group Design

10.1K
The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between...
10.1K
Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs01:15

Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs

162
Body:Bioequivalence experimental study designs play a pivotal role in testing the effectiveness of various treatments. Key among these are the repeated measures, cross-over, carry-over, and Latin square designs. In the repeated measures design, each subject receives all treatments, allowing for temporal comparisons. This type of design is useful in reducing variability but requires careful planning to avoid bias.The cross-over design, an economical method, involves sequential administration of...
162
Longitudinal Research02:20

Longitudinal Research

13.0K
Sometimes we want to see how people change over time, as in studies of human development and lifespan. When we test the same group of individuals repeatedly over an extended period of time, we are conducting longitudinal research. Longitudinal research is a research design in which data-gathering is administered repeatedly over an extended period of time. For example, we may survey a group of individuals about their dietary habits at age 20, retest them a decade later at age 30, and then again...
13.0K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

People in more individualist cultures are more motivated to make others feel better.

Proceedings of the National Academy of Sciences of the United States of America·2026
Same author

Individual Differences in Great Ape Cognition Across Time and Domains: Stability, Structure, and Predictability.

Psychological science·2026
Same author

Modeling within-level latent interaction effects in multilevel vector-autoregressive models.

Behavior research methods·2025
Same author

Unique Contributions of Dynamic Affect Indicators - Beyond Static Variability.

Multivariate behavioral research·2025
Same author

Testing measurement and structural invariance in latent mediation models - A comparison of IPCR and Bayesian MNLFA.

Behavior research methods·2025
Same author

Tearjerkers may leave some eyes dry: Emotional reactivity to film clips from adolescence to old age.

The British journal of developmental psychology·2025

Related Experiment Video

Updated: Jan 8, 2026

Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity
08:40

Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity

Published on: June 12, 2019

7.8K

Idiographic interrater reliability measures for intensive longitudinal multirater data.

Tobias Koch1, Miriam F Jaehne1, Michaela Riediger1

  • 1Friedrich-Schiller-Universität Jena, Jena, Germany.

The British Journal of Mathematical and Statistical Psychology
|December 20, 2025
PubMed
Summary
This summary is machine-generated.

This study introduces a new model for assessing agreement between different types of raters in psychological research. Relationship duration and partner cognitive resources enhance rater consistency for innovations.

Keywords:
empathic accuracyidiographic rater consistencylongitudinal structural equation modelingmultirater analysistime series analysis

More Related Videos

Author Spotlight: Assessing the Reliability of Doppler Ultrasound in Measuring Leg Blood Flow
09:18

Author Spotlight: Assessing the Reliability of Doppler Ultrasound in Measuring Leg Blood Flow

Published on: December 15, 2023

3.4K
A Protocol of Manual Tests to Measure Sensation and Pain in Humans
07:28

A Protocol of Manual Tests to Measure Sensation and Pain in Humans

Published on: December 19, 2016

21.6K

Related Experiment Videos

Last Updated: Jan 8, 2026

Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity
08:40

Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity

Published on: June 12, 2019

7.8K
Author Spotlight: Assessing the Reliability of Doppler Ultrasound in Measuring Leg Blood Flow
09:18

Author Spotlight: Assessing the Reliability of Doppler Ultrasound in Measuring Leg Blood Flow

Published on: December 15, 2023

3.4K
A Protocol of Manual Tests to Measure Sensation and Pain in Humans
07:28

A Protocol of Manual Tests to Measure Sensation and Pain in Humans

Published on: December 19, 2016

21.6K

Area of Science:

  • Psychology
  • Quantitative Psychology
  • Psychometrics

Background:

  • Interrater reliability is essential in psychological research.
  • Existing models often struggle with structurally different rater types (e.g., self vs. partner reports).
  • Need for methods to assess person-specific rater consistency and its predictors.

Purpose of the Study:

  • Propose a novel multilevel latent time series model (MR-MLTS) for intensive longitudinal data with multiple, structurally different raters.
  • Enable estimation of idiographic (person-specific) rater consistency coefficients.
  • Facilitate linking rater consistency to external variables.

Main Methods:

  • Developed a multilevel latent time series model (MR-MLTS).
  • Implemented the model in Mplus and a new R package (mlts).
  • Applied the model to intensive longitudinal data from 100 heterosexual couples (86 time points).

Main Results:

  • Relationship duration and partner cognitive resources positively predict rater consistency for innovations.
  • Simulation results highlight the importance of the number of time points for estimating idiographic coefficients.
  • Number of participants is crucial for recovering random effect variances.

Conclusions:

  • The MR-MLTS model offers a flexible approach to quantifying rater agreement in complex dyadic and longitudinal studies.
  • Findings underscore the role of relationship factors in shaping rater consistency.
  • The model provides valuable insights into measurement precision and individual differences in agreement.