Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Distribution Reliability and Automation01:25

Distribution Reliability and Automation

107
Distribution reliability in electrical power systems is critical for ensuring an uninterrupted power supply to consumers at minimal cost. According to IEEE Standard Terms, reliability is the probability that a device will function without failure over a specified time period or amount of usage. For electric power distribution, this translates to maintaining continuous power supply and addressing customer concerns over power outages. Several indices, as defined by IEEE Standard 1366-2012, are...
107
Reliability and Validity01:29

Reliability and Validity

12.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.7K
Survival Tree01:19

Survival Tree

75
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
 Building a Survival Tree
Constructing a...
75
Statistical Analysis: Overview01:11

Statistical Analysis: Overview

6.3K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
6.3K
Confidence Coefficient01:24

Confidence Coefficient

7.6K
The confidence coefficient is also known as the confidence level or degree of confidence. It is the percent expression for the probability, 1-α, that the confidence interval contains the true population parameter assuming that the confidence interval is obtained after sufficient unbiased sampling; for example, if the CL = 90%, then in 90 out of 100 samples the interval estimate will enclose the true population parameter. Here α is the area under the curve, distributed equally under...
7.6K
Residuals and Least-Squares Property01:11

Residuals and Least-Squares Property

7.3K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.3K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Depth of neutrophil mobilization stratifies survival in ST-elevation myocardial infarction.

Nature cardiovascular research·2026
Same author

CD177 Deficiency Defines a Stable Subtype of Human Neutrophil Granulocytes with Tumor Promoting Activity.

Advanced science (Weinheim, Baden-Wurttemberg, Germany)·2026
Same author

Development of a long-acting relaxin for the treatment of pulmonary hypertension.

British journal of pharmacology·2026
Same author

Effects of Engaging Fathers and Bundling Nutrition and Parenting Interventions on Child Nutrition Outcomes and Related Parent Behaviors in Rural Tanzania: Results From EFFECTS, a Five-Arm Cluster-Randomized Controlled Trial.

Maternal & child nutrition·2026
Same author

BRG1 (SMARCA4) Status Dictates the Response to EGFR Inhibitors in Wild-Type EGFR Non-Small Cell Lung Cancer.

Cancers·2026
Same author

SARS-CoV-2 peptide fragments selectively dysregulate specific immune cell populations via Gaussian curvature targeting.

Proceedings of the National Academy of Sciences of the United States of America·2026

Related Experiment Video

Updated: Jun 19, 2025

An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.0K

Quantitative assessment of machine learning reliability and resilience.

Zakaria Faddi1, Karen da Mata1, Priscila Silva1

  • 1Department of Electrical and Computer Engineering, University of Massachusetts Dartmouth, Dartmouth, Massachusetts, USA.

Risk Analysis : an Official Publication of the Society for Risk Analysis
|July 23, 2024
PubMed
Summary

This study introduces quantitative methods to assess machine learning (ML) system reliability and resilience over time. Software reliability and resilience models accurately track ML performance degradation and improvement, aiding engineers in risk assessment.

Keywords:
adversarial attacksmachine learningresiliencesoftware reliability

More Related Videos

Machine Learning Algorithms for Early Detection of Bone Metastases in an Experimental Rat Model
07:15

Machine Learning Algorithms for Early Detection of Bone Metastases in an Experimental Rat Model

Published on: August 16, 2020

6.8K
Design and Analysis for Fall Detection System Simplification
08:05

Design and Analysis for Fall Detection System Simplification

Published on: April 6, 2020

10.6K

Related Experiment Videos

Last Updated: Jun 19, 2025

An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.0K
Machine Learning Algorithms for Early Detection of Bone Metastases in an Experimental Rat Model
07:15

Machine Learning Algorithms for Early Detection of Bone Metastases in an Experimental Rat Model

Published on: August 16, 2020

6.8K
Design and Analysis for Fall Detection System Simplification
08:05

Design and Analysis for Fall Detection System Simplification

Published on: April 6, 2020

10.6K

Area of Science:

  • Computer Science
  • Artificial Intelligence
  • Software Engineering

Background:

  • Machine learning (ML) models are increasingly used in safety-critical applications, facing dynamic and hostile environments.
  • Ensuring the reliability and resilience of ML systems is crucial for real-world deployment.
  • Existing research often focuses on improving ML robustness but lacks quantitative methods for assessing system changes over time.

Purpose of the Study:

  • To address the gap in quantitative assessment of ML system reliability and resilience.
  • To demonstrate a method for collecting data during ML training and testing for reliability and resilience modeling.
  • To provide a framework for quantitative risk assessment and performance tracking of ML systems.

Main Methods:

  • Collected data during ML model training and testing, suitable for software reliability and resilience modeling.
  • Applied software reliability models with and without covariates.
  • Utilized a resilience model based on multiple linear regression with covariate interactions.
  • Illustrated the approach using an image recognition model subjected to adversarial attacks and retraining.

Main Results:

  • Software reliability models incorporating covariates provided a more accurate characterization of the misclassification discovery process.
  • The resilience model, including covariate interactions, best tracked and predicted performance degradation and recovery.
  • The proposed methods enable quantitative tracking and prediction of ML model performance changes.

Conclusions:

  • Software reliability and resilience models offer rigorous quantitative assurance for ML-enabled systems.
  • The demonstrated approach facilitates objective comparison of different ML training and testing strategies.
  • Quantitative assessment is vital for managing risks associated with ML in safety-critical domains.