Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation01:24

One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation

589
This lesson introduces two critical methods in pharmacokinetics, the Wagner-Nelson and Loo-Riegelman methods, used for estimating the absorption rate constant (ka) for drugs administered via non-intravenous routes. The Wagner-Nelson method relates ka to the plasma concentration derived from the slope of a semilog percent unabsorbed time plot. However, it is limited to drugs with one-compartment kinetics and can be impacted by factors like gastrointestinal motility or enzymatic degradation.
On...
589
Accuracy, limits, and approximation01:28

Accuracy, limits, and approximation

484
Accuracy, limits, and approximations are common in many fields, especially in engineering calculations. These concepts are imperative for ensuring that a given value is as close as possible to its true value.
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
484
Accuracy and Errors in Hypothesis Testing01:13

Accuracy and Errors in Hypothesis Testing

232
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
232
Estimating Population Mean with Unknown Standard Deviation01:22

Estimating Population Mean with Unknown Standard Deviation

8.2K
In practice, we rarely know the population standard deviation. In the past, when the sample size was large, this did not present a problem to statisticians. They used the sample standard deviation s as an estimate for σ and proceeded as before to calculate a confidence interval with close enough results. However, statisticians ran into problems when the sample size was small. A small sample size caused inaccuracies in the confidence interval.
William S. Gosset (1876–1937) of the...
8.2K
Estimating Population Mean with Known Standard Deviation01:16

Estimating Population Mean with Known Standard Deviation

8.9K
To construct a confidence interval for a single unknown population mean μ, where the population standard deviation is known, we need sample mean as an estimate for μ and we need the margin of error. Here, the margin of error (EBM) is called the error bound for a population mean (abbreviated EBM). The sample mean is the point estimate of the unknown population mean μ.
The confidence interval estimate will have the form as follows:
(point estimate - error bound, point estimate +...
8.9K
Mechanistic Models: Compartment Models in Individual and Population Analysis01:23

Mechanistic Models: Compartment Models in Individual and Population Analysis

67
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
67

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Penalized eigenvalue block averaging: Extension to nested model comparison and Monte Carlo evaluations.

Behavior research methods·2026
Same author

Inference for Disattenuated Correlations.

Applied psychological measurement·2026
Same author

Measures of Agreement with Multiple Raters: Fréchet Variances and Inference.

Psychometrika·2024
Same author

Partial Identification of Latent Correlations with Ordinal Data.

Psychometrika·2023
Same author

Modelling publication bias and p-hacking.

Biometrics·2021
Same author

Partial Identification of Latent Correlations with Binary Data.

Psychometrika·2020
Same journal

BAYESIAN MIXED MULTIDIMENSIONAL SCALING FOR AUDITORY PROCESSING.

Psychometrika·2026
Same journal

Testing linear hypotheses in repeated measures generalized linear models using external information.

Psychometrika·2026
Same journal

When Do Unifactorial Items Increase the Reliability?

Psychometrika·2026
Same journal

Longitudinal Designs for Diagnostic Models: Identification and Estimation.

Psychometrika·2026
Same journal

Modeling Rare Events and Nonmonotone Nonignorable Missingness of Time-Varying Outcomes and Predictors in Binary Time-Series Daily Diary Data: A Bayesian Selection Model.

Psychometrika·2026
Same journal

Revelle's Beta: The Wait Is Over-Computation Becomes Possible.

Psychometrika·2026
See all related articles

Related Experiment Video

Updated: Jul 27, 2025

The Spatial Memory Game: Testing the Relationship Between Spatial Language, Object Knowledge, and Spatial Cognition
05:15

The Spatial Memory Game: Testing the Relationship Between Spatial Language, Object Knowledge, and Spatial Cognition

Published on: February 19, 2018

10.9K

Measuring Agreement Using Guessing Models and Knowledge Coefficients.

Jonas Moss1

  • 1Department of Data Science and Analytics, BI Norwegian Business School, Oslo, Norway. jonas.moss@bi.no.

Psychometrika
|June 8, 2023
PubMed
Summary
This summary is machine-generated.

This study introduces guessing models and a knowledge coefficient for measuring inter-rater agreement. The Brennan-Prediger coefficient demonstrated superior performance in simulations, offering better coverage in challenging conditions.

Keywords:
AC1AgreementCohen’s kappaInterrater reliability

More Related Videos

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
08:12

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Published on: March 1, 2022

2.6K
Probing the Limits of Egg Recognition Using Egg Rejection Experiments Along Phenotypic Gradients
07:34

Probing the Limits of Egg Recognition Using Egg Rejection Experiments Along Phenotypic Gradients

Published on: August 22, 2018

8.3K

Related Experiment Videos

Last Updated: Jul 27, 2025

The Spatial Memory Game: Testing the Relationship Between Spatial Language, Object Knowledge, and Spatial Cognition
05:15

The Spatial Memory Game: Testing the Relationship Between Spatial Language, Object Knowledge, and Spatial Cognition

Published on: February 19, 2018

10.9K
A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
08:12

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Published on: March 1, 2022

2.6K
Probing the Limits of Egg Recognition Using Egg Rejection Experiments Along Phenotypic Gradients
07:34

Probing the Limits of Egg Recognition Using Egg Rejection Experiments Along Phenotypic Gradients

Published on: August 22, 2018

8.3K

Area of Science:

  • Statistics
  • Psychometrics
  • Data Analysis

Background:

  • Existing agreement measures like Cohen's kappa and Fleiss' kappa rely on specific rating models.
  • A unified framework is needed to encompass various explicit models of judge rating behavior.

Purpose of the Study:

  • To propose a general class of models, termed 'guessing models,' for understanding how judges make ratings.
  • To introduce a unified measure of agreement, the 'knowledge coefficient,' associated with these models.
  • To evaluate the performance of different agreement measures, including the proposed coefficient, under various conditions.

Main Methods:

  • Development of the 'guessing model' framework to generalize existing rating models.
  • Derivation of the 'knowledge coefficient' as a measure of agreement within this framework.
  • Estimation of the knowledge coefficient and analysis of its asymptotic distributions.
  • Conducting sensitivity analysis and simulation studies to compare confidence interval coverage.

Main Results:

  • The knowledge coefficient unifies several existing agreement measures under specific assumptions.
  • The Brennan-Prediger coefficient was found to be a robust estimator of the knowledge coefficient.
  • Simulation results indicated that the Brennan-Prediger coefficient offers superior confidence interval coverage, especially in unfavorable circumstances.

Conclusions:

  • Guessing models provide a flexible framework for agreement analysis.
  • The Brennan-Prediger coefficient is recommended for its robust performance and better coverage in inter-rater reliability studies.
  • The knowledge coefficient offers a unified approach to agreement measurement, adaptable to different judge rating models.