Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Kendall's Coefficient of Concordance01:20

Kendall's Coefficient of Concordance

527
Kendall's Coefficient of Concordance (W), also known as Kendall's W, is a non-parametric statistical measure used to assess the agreement or concordance between multiple raters or judges when they rank a set of items. It is often used when you have ordinal data (ranks) and you want to see if there is consistency or consensus among the raters. It is widely applied in research areas such as psychology, medicine, and social sciences, where multiple judges are asked to rank or rate subjects...
527
Test for Homogeneity01:23

Test for Homogeneity

2.1K
The goodness–of–fit test can be used to decide whether a population fits a given distribution, but it will not suffice to decide whether two populations follow the same unknown distribution. A different test, called the test for homogeneity, can be used to conclude whether two populations have the same distribution. To calculate the test statistic for a test for homogeneity, follow the same procedure as with the test of independence. The hypotheses for the test for homogeneity can...
2.1K
Bonferroni Test01:10

Bonferroni Test

2.8K
The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
2.8K
Introduction to Test of Independence01:21

Introduction to Test of Independence

2.4K
In statistics, the term independence means that one can directly obtain the probability of any event involving both variables by multiplying their individual probabilities. Tests of independence are chi-square tests involving the use of a contingency table of observed (data) values.
The test statistic for a test of independence is similar to that of a goodness-of-fit test:
2.4K
Spearman's Rank Correlation Test01:20

Spearman's Rank Correlation Test

1.0K
Spearman's rank correlation test, also known as Spearman's rho, is a nonparametric method for assessing the strength and direction of association between two variables. This test is particularly valuable when the data distribution is unknown or when the assumption of normality does not hold. Named after the English psychologist and statistician Dr. Charles Edward Spearman, it serves as the nonparametric counterpart to Pearson's correlation coefficient.
Spearman's test calculates...
1.0K
Sign Test for Matched Pairs01:17

Sign Test for Matched Pairs

207
The sign test for matched pairs offers a robust method for comparing two paired samples, often for the effects of an intervention in one of them. This method is very useful in situations where the underlying distribution of the data is unknown. The test compares two related samples—often pre- and post-treatment measurements on the same subjects—to determine if there are significant differences in their median values.
To conduct the sign test, we first calculate the differences in...
207

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

An Alternative Treatment Effect Measure for Time-to-Event Oncology Randomized Trials.

Cancers·2025
Same author

Optimizing One-Sample Tests for Proportions in Single- and Two-Stage Oncology Trials.

Cancers·2025
Same author

Strategies to boost statistical efficiency in randomized oncology trials with primary time-to-event endpoints.

Statistical methods in medical research·2025
Same author

Consequences of the perivascular niche remodeling for tumoricidal T-cell trafficking into metastasis of ovarian cancer.

Research square·2024
Same author

Multiomics integrated analysis and experimental validation identify TLR4 and ALOX5 as oxidative stress-related biomarkers in intracranial aneurysms.

Journal of neuroinflammation·2024
Same author

Asymmetric catalytic concise synthesis of 3-(3-indolomethyl)-oxindoles for the construction of trigolute analogs.

Chemical communications (Cambridge, England)·2024

Related Experiment Video

Updated: Sep 10, 2025

Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity
08:40

Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity

Published on: June 12, 2019

7.5K

Robust Permutation Test of Intraclass Correlation Coefficient for Assessing Agreement.

Mengyu Fang1, Alan David Hutson1, Han Yu1

  • 1Department of Biostatistics and Bioinformatics, Roswell Park Comprehensive Cancer Center, Buffalo, NY 14263, USA.

Cancers
|August 28, 2025
PubMed
Summary
This summary is machine-generated.

A new studentized permutation test reliably controls errors when assessing inter-rater reliability using the intraclass correlation coefficient (ICC) in oncology. This robust method ensures accurate measurements for critical clinical and research decisions.

Keywords:
ICCagreementpermutation test

More Related Videos

Author Spotlight: Validation of SICOLE-R for Assessing Cognitive and Reading Skills in Spanish-Speaking Children and Its Role in Personalized Education
09:00

Author Spotlight: Validation of SICOLE-R for Assessing Cognitive and Reading Skills in Spanish-Speaking Children and Its Role in Personalized Education

Published on: August 16, 2024

908
How to Calculate and Validate Inter-brain Synchronization in a fNIRS Hyperscanning Study
05:33

How to Calculate and Validate Inter-brain Synchronization in a fNIRS Hyperscanning Study

Published on: September 8, 2021

6.7K

Related Experiment Videos

Last Updated: Sep 10, 2025

Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity
08:40

Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity

Published on: June 12, 2019

7.5K
Author Spotlight: Validation of SICOLE-R for Assessing Cognitive and Reading Skills in Spanish-Speaking Children and Its Role in Personalized Education
09:00

Author Spotlight: Validation of SICOLE-R for Assessing Cognitive and Reading Skills in Spanish-Speaking Children and Its Role in Personalized Education

Published on: August 16, 2024

908
How to Calculate and Validate Inter-brain Synchronization in a fNIRS Hyperscanning Study
05:33

How to Calculate and Validate Inter-brain Synchronization in a fNIRS Hyperscanning Study

Published on: September 8, 2021

6.7K

Area of Science:

  • Statistics
  • Oncology
  • Biostatistics

Background:

  • Inter-rater reliability is crucial in oncology for consistent measurements, impacting clinical and research decisions.
  • The intraclass correlation coefficient (ICC) is a key statistic for assessing agreement between raters.
  • Accurate assessment of ICC is vital for reliable biomarker and tumor size evaluations.

Purpose of the Study:

  • To develop and validate a robust statistical test for hypothesis testing of the ICC(2,1) with two raters.
  • To address the limitations of naive permutation tests in controlling type I error rates for ICC.
  • To provide a reliable method for assessing inter-rater agreement in oncology.

Main Methods:

  • Evaluation of a naive permutation test for ICC(2,1) hypothesis testing.
  • Development of a novel, robust permutation test utilizing a studentized statistic.
  • Asymptotic validity proof for the studentized test, even with dependent variables.

Main Results:

  • The naive permutation test demonstrated unreliable type I error control.
  • The proposed studentized permutation test consistently maintained type I error control across simulations.
  • The new test showed superior performance, especially with small sample sizes.

Conclusions:

  • The studentized permutation test provides a statistically valid and robust approach for ICC(2,1) assessment.
  • This method ensures reliable inter-rater reliability analysis in oncology.
  • The test demonstrated practical utility in real-world oncology datasets.