相关实验视频
Updated: Jun 19, 2025
05:37
An R-Based Landscape Validation of a Competing Risk Model
Published on: September 16, 2022
2.0K
对机器学习可靠性和弹性进行定量评估
Zakaria Faddi1, Karen da Mata1, Priscila Silva1
1Department of Electrical and Computer Engineering, University of Massachusetts Dartmouth, Dartmouth, Massachusetts, USA.
概括
本研究引入了定量方法来评估机器学习 (ML) 系统的可靠性和随时间的弹性. 软件可靠性和弹性模型准确地追踪ML性能恶化和改进,帮助工程师进行风险评估.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 软件工程 软件工程 软件工程
背景情况:
- 机器学习 (ML) 模型越来越多地用于安全关键的应用,面对动态和敌对的环境.
- 确保ML系统的可靠性和弹性对于现实世界的部署至关重要.
- 现有的研究往往侧重于提高ML的稳定性,但缺乏用于评估系统随时间变化的定量方法.
研究的目的:
- 解决ML系统可靠性和弹性量化评估的差距.
- 展示在ML训练和可靠性和弹性建模测试期间收集数据的方法.
- 为ML系统的定量风险评估和性能跟踪提供一个框架.
主要方法:
- 在ML模型培训和测试期间收集的数据,适合软件可靠性和弹性建模.
- 应用软件可靠性模型,有或没有共变量.
- 利用基于多重线性回归与共变相互作用的弹性模型.
- 用图像识别模型来说明这种方法,该模型经过对抗攻击和再培训.
主要成果:
- 包含共变量的软件可靠性模型为错误分类发现过程提供了更准确的描述.
- 弹性模型,包括共同变量相互作用,最好跟踪和预测性能退化和恢复.
- 提出的方法允许定量跟踪和预测ML模型性能变化.
结论:
- 软件可靠性和弹性模型为支持机器学习的系统提供了严格的定量保证.
- 证明的方法使不同ML培训和测试策略的客观比较更容易.
- 量化评估对于管理安全关键领域的ML相关风险至关重要.
相关概念视频
01:25Distribution Reliability and Automation
107
Distribution reliability in electrical power systems is critical for ensuring an uninterrupted power supply to consumers at minimal cost. According to IEEE Standard Terms, reliability is the probability that a device will function without failure over a specified time period or amount of usage. For electric power distribution, this translates to maintaining continuous power supply and addressing customer concerns over power outages. Several indices, as defined by IEEE Standard 1366-2012, are...
107
01:29Reliability and Validity
12.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.7K
01:19Survival Tree
75
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
75
01:11Statistical Analysis: Overview
6.3K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
6.3K
01:24Confidence Coefficient
7.6K
The confidence coefficient is also known as the confidence level or degree of confidence. It is the percent expression for the probability, 1-α, that the confidence interval contains the true population parameter assuming that the confidence interval is obtained after sufficient unbiased sampling; for example, if the CL = 90%, then in 90 out of 100 samples the interval estimate will enclose the true population parameter. Here α is the area under the curve, distributed equally under...
7.6K
01:11Residuals and Least-Squares Property
7.3K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.3K