Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Interpretation of Confidence Intervals

Interpretation of Confidence Intervals

A confidence interval is a better estimate of the population than a point estimate, as it uses a range of values from a sample instead of a single value.
Confidence intervals have confidence coefficients that are crucial for their interpretation. The most common confidence coefficients are 0.90, 0.95, and 0.99, which can be written as percentages–90%, 95%, and 99%, respectively.
Suppose a person calculates a confidence interval with a confidence coefficient of 0.95. In that case, they can...

Reliability and Validity

Reliability and Validity

Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.

Accuracy and Errors in Hypothesis Testing

Accuracy and Errors in Hypothesis Testing

Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...

Testing a Claim about Population Proportion

Testing a Claim about Population Proportion

A complete procedure for testing a claim about a population proportion is provided here.
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...

Confidence Intervals

Confidence Intervals

An unbiased point estimate is often insufficient to predict a population estimate, such as population mean or population proportion. In this scenario, a confidence interval is used. A confidence interval is an estimate similar to a sample proportion. However, unlike the point estimate which is a single value, the confidence interval contains a range of values. These values have lower and upper limits, known as confidence limits, and can be designated as L1 and L2, respectively.
A confidence...

Introduction to Test of Independence

Introduction to Test of Independence

In statistics, the term independence means that one can directly obtain the probability of any event involving both variables by multiplying their individual probabilities. Tests of independence are chi-square tests involving the use of a contingency table of observed (data) values.
The test statistic for a test of independence is similar to that of a goodness-of-fit test:

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

When Do Unifactorial Items Increase the Reliability?

Psychometrika·2026

Same author

Bias and precision in true-score estimation.

The British journal of mathematical and statistical psychology·2026

Same author

Recognize the Value of the Sum Score, Psychometrics' Greatest Accomplishment.

Psychometrika·2026

Same author

Proof of Reliability Convergence to 1 at Rate of Spearman-Brown Formula for Random Test Forms and Irrespective of Item Pool Dimensionality.

Psychometrika·2026

Same author

Reliability Theory for Measurements with Variable Test Length, Illustrated with ERN and Pe Collected in the Flanker Task.

Psychometrika·2026

Same author

Rejoinder to McNeish and Mislevy: What Does Psychological Measurement Require?

Psychometrika·2026

Search research articles

Related Experiment Video

Updated: May 11, 2026

Doppler Ultrasound-Based Leg Blood Flow Assessment During Single-Leg Knee-Extensor Exercise in an Uncontrolled Setting

Doppler Ultrasound-Based Leg Blood Flow Assessment During Single-Leg Knee-Extensor Exercise in an Uncontrolled Setting

Published on: December 15, 2023

Probability interpretations of intraclass reliabilities.

Jules L Ellis¹

¹Ellis Statistical Consultations, Grotestraat 63, 6511 VB Nijmegen, the Netherlands; School of Psychology and Artificial Intelligence, Radboud University Nijmegen, P.O.B. 9104, 6500 HE Nijmegen, the Netherlands.

Statistics in Medicine

|May 25, 2013

Summary

Reliability in organizational ratings, often measured by intraclass correlations, can be understood through easier-to-grasp probabilities. These probabilities reveal an inverse relationship between classification correctness and informativeness, aiding in setting acceptable reliability levels.

Keywords:

bivariate normal distribution decision probability intraclass correlation public health care rating

More Related Videos

Advancing Dyslexia Assessment in Children Through Computerized Testing

Advancing Dyslexia Assessment in Children Through Computerized Testing

Published on: August 16, 2024

Related Experiment Videos

Last Updated: May 11, 2026

Doppler Ultrasound-Based Leg Blood Flow Assessment During Single-Leg Knee-Extensor Exercise in an Uncontrolled Setting

Doppler Ultrasound-Based Leg Blood Flow Assessment During Single-Leg Knee-Extensor Exercise in an Uncontrolled Setting

Published on: December 15, 2023

Advancing Dyslexia Assessment in Children Through Computerized Testing

Advancing Dyslexia Assessment in Children Through Computerized Testing

Published on: August 16, 2024

Area of Science:

Statistics
Psychometrics
Organizational Behavior

Background:

Many studies rating organizations use intraclass correlations for reliability.
Consumers of statistical data (e.g., patients, policymakers) may lack expertise to judge reliability standards.
Existing reliability metrics can be abstract for non-statistical audiences.

Purpose of the Study:

To relate intraclass correlation reliability to more interpretable probabilities.
To provide a framework for understanding and setting acceptable reliability levels in organizational research.
To clarify the trade-offs between classification informativeness and correctness.

Main Methods:

Demonstration of the relationship between intraclass correlation reliability and derived probabilities.
Conceptualization of probabilities as informativeness (proportion above/below mean) and correctness (accurate classification).
Analysis of the inverse relationship between informativeness and correctness for a given reliability.

Main Results:

Reliability is directly linked to the probability of organizations being classified significantly above or below the mean.
Reliability is also linked to the probability of correct classification, given a significant classification.
An inverse relationship exists: increasing correctness decreases informativeness, and vice versa, for a fixed reliability.

Conclusions:

Interpretable probabilities can supplement intraclass correlations for assessing reliability.
Understanding the trade-off between informativeness and correctness aids in determining appropriate reliability thresholds.
This approach can enhance the practical application and interpretation of reliability estimates in organizational studies.