Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Calibration Curves: Linear Least Squares01:20

Calibration Curves: Linear Least Squares

A calibration curve is a plot of the instrument's response against a series of known concentrations of a substance. This curve is used to set the instrument response levels, using the substance and its concentrations as standards. Alternatively, or additionally, an equation is fitted to the calibration curve plot and subsequently used to calculate the unknown concentrations of other samples reliably.
For data that follow a straight line, the standard method for fitting is the linear...
Instrument Calibration01:12

Instrument Calibration

Instrument calibration is essential for ensuring that instruments produce accurate and consistent results. It is vital in manufacturing, healthcare, testing laboratories, and scientific research. Calibration processes are specific to each instrument and help enhance data accuracy. Each instrument has a unique calibration process tailored to its design and function to improve data accuracy.
Analytical Balance Calibration
An analytical balance measures mass and requires regular calibration to...
Calibration Curves: Correlation Coefficient01:10

Calibration Curves: Correlation Coefficient

In a linear calibration curve, there is a value called the calibration coefficient, denoted by 'r,' which measures the strength and the direction of association between two variables. The correlation coefficient value ranges from −1 to +1. A value of +1 indicates a perfect positive linear correlation, −1 denotes a perfect negative correlation, and 0 implies no correlation between the two variables. A positive correlation value establishes that as one variable increases, the other increases, and...
Classification of Systems-I01:26

Classification of Systems-I

Linearity is a system property characterized by a direct input-output relationship, combining homogeneity and additivity.
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
Classification of Systems-II01:31

Classification of Systems-II

Continuous-time systems have continuous input and output signals, with time measured continuously. These systems are generally defined by differential or algebraic equations. For instance, in an RC circuit, the relationship between input and output voltage is expressed through a differential equation derived from Ohm's law and the capacitor relation,
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Using concept hierarchies to improve calculation of patient similarity.

Journal of biomedical informatics·2016
Same author

cFinder: definition and quantification of multiple haplotypes in a mixed sample.

BMC research notes·2015
Same author

A prospective study of mobile phones for dermatology in a clinical setting.

Journal of telemedicine and telecare·2013
Same author

MALDI-MS tissue imaging identification of biliverdin reductase B overexpression in prostate cancer.

Journal of proteomics·2013
Same author

The CHOP postnatal weight gain, birth weight, and gestational age retinopathy of prematurity risk model.

Archives of ophthalmology (Chicago, Ill. : 1960)·2012
Same author

Differences in examination characteristics of pigmented skin lesions: results of an eye tracking study.

Artificial intelligence in medicine·2012
Same journal

Sensitivity Analyses of a Scoring System for a Contraception Decision Aid.

AMIA ... Annual Symposium proceedings. AMIA Symposium·2026
Same journal

Improving electronic health record processing of large language models via retrieval-augmented generation: A case study on dietary supplements.

AMIA ... Annual Symposium proceedings. AMIA Symposium·2026
Same journal

Developing a User-Centered Mobile Application Prototype: Bridging Lower-Limb Fracture Care from Skilled Nursing Facility and Back to the Community.

AMIA ... Annual Symposium proceedings. AMIA Symposium·2026
Same journal

KERAP: A Knowledge-Enhanced Reasoning Approach for Accurate Zero-shot Diagnosis Prediction Using Multi-agent LLMs.

AMIA ... Annual Symposium proceedings. AMIA Symposium·2026
Same journal

Automating Adjudication of Cardiovascular Events Using Large Language Models.

AMIA ... Annual Symposium proceedings. AMIA Symposium·2026
Same journal

Predictive Factors and State-Level Barriers to Postpartum Birth Control Usage in the United States: Insights from PRAMS Phase 8.

AMIA ... Annual Symposium proceedings. AMIA Symposium·2026
See all related articles

Related Experiment Video

Updated: May 15, 2026

Construction of Models for Nondestructive Prediction of Ingredient Contents in Blueberries by Near-infrared Spectroscopy Based on HPLC Measurements
10:25

Construction of Models for Nondestructive Prediction of Ingredient Contents in Blueberries by Near-infrared Spectroscopy Based on HPLC Measurements

Published on: June 28, 2016

Testing the calibration of classification models from first principles.

Stephan Dreiseitl1, Melanie Osl

  • 1Dept. of Software Engineering, Upper Austria University of Applied Sciences, Hagenberg, Austria.

AMIA ... Annual Symposium Proceedings. AMIA Symposium
|January 11, 2013
PubMed
Summary
This summary is machine-generated.

We introduce a novel statistical test for assessing classification model calibration. This new method directly evaluates the probability of observed labels, offering a more principled approach than existing goodness-of-fit tests.

Related Experiment Videos

Last Updated: May 15, 2026

Construction of Models for Nondestructive Prediction of Ingredient Contents in Blueberries by Near-infrared Spectroscopy Based on HPLC Measurements
10:25

Construction of Models for Nondestructive Prediction of Ingredient Contents in Blueberries by Near-infrared Spectroscopy Based on HPLC Measurements

Published on: June 28, 2016

Area of Science:

  • Machine Learning
  • Statistical Modeling
  • Model Calibration

Background:

  • Accurate calibration assessment for classification models is challenging due to the lack of a gold standard.
  • Current methods often rely on grouping probabilities and using goodness-of-fit tests like chi-squared.
  • Existing approaches have limitations in directly evaluating model output accuracy.

Purpose of the Study:

  • To propose a novel, first-principles approach for testing the calibration of classification models.
  • To develop a statistical test that directly assesses the accuracy of predicted probabilities.
  • To provide a more robust alternative to established calibration assessment methods.

Main Methods:

  • Developed a new calibration test based on statistical hypothesis testing principles.
  • Formulated the null hypothesis that model outputs accurately estimate true class probabilities.
  • Calculated a p-value by assessing the improbability of observed class labels under the null hypothesis.

Main Results:

  • The proposed test provides a direct assessment of model calibration.
  • Experimental results show the new test performs comparably to the Hosmer-Lemeshow test.
  • In some cases, the proposed test demonstrates superior performance compared to the standard method.

Conclusions:

  • The novel statistical test offers a principled and effective method for classification model calibration assessment.
  • This approach addresses limitations of existing goodness-of-fit tests.
  • The new test is a valuable tool for ensuring reliable model predictions in machine learning.