在表格数据上的二进制分类中寻求机器学习模型的可靠性
Vitor Cirilo Araujo Santos1,2, Lucas Cardoso3,4, Ronnie Alves3,4
1Federal University of Pará, PPGCC, Belém, 66075-110, Brazil. vitor.cirilo3@gmail.com.
Scientific reports
|October 27, 2023
概括
本研究引入了一种使用物件响应理论的新方法,用于评估超出简单准确度指标的机器学习 (ML) 模型可靠性. 它有助于识别不可靠的环境,确保对ML应用程序进行更好的概括.
科学领域:
- 机器学习 机器学习
- 人工智能的人工智能
- 心理测量 心理测量 心理测量
背景情况:
- 目前的机器学习 (ML) 模型评估指标,如精度和F1分数,主要评估输出准确性.
- 这些指标不评估模型的基础学习过程或其上下文理解.
- 这种限制可以导致模型在训练中表现良好,但在通用化过程中失败.
研究的目的:
- 提出一种新的方法来评估机器学习模型的可靠性.
- 提供一个超越传统准确性指标的评估框架.
- 加强机器学习模型的验证过程,确保更好的上下文概括.
主要方法:
- 开发一种基于物品响应理论 (IRT) 的方法.
- 应用IRT原则来分析机器学习模型中的上下文可靠性.
- 与传统评估程序进行比较分析.
主要成果:
- 提出的基于项目响应理论的方法有效地识别了机器学习模型中不可靠的环境.
- 这种方法提供了一个独特的验证层,补充现有指标.
- 它提供了关于模型是否真正学习了有意义的上下文元素的见解.
结论:
- 对于机器学习模型的传统评估指标不足以评估上下文可靠性.
- 基于项目响应理论的方法提供了一种可靠的方式来验证ML模型的学习和概括.
- 这种新方法对于开发更可靠,更具上下文意识的机器学习系统至关重要.
相关概念视频
Survival Tree
88
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
88
How Data are Classified: Categorical Data
33.8K
A variable, usually notated by capital letters such as X and Y, is a characteristic or measurement that can be determined for each member of a population. Data are the actual values of variables. They may be numbers, or they may be words. Datum is a single value.
Data are classified based on whether they are measurable or not. Categorical data cannot be measured; instead, it can be divided into categories. For example, if Y denotes a person's party affiliation, some examples of Y include...
Data are classified based on whether they are measurable or not. Categorical data cannot be measured; instead, it can be divided into categories. For example, if Y denotes a person's party affiliation, some examples of Y include...
33.8K
How Data are Classified: Numerical Data
28.9K
Data that are countable or measurable in specific units are called numerical or quantitative data. Quantitative data are always numbers. Quantitative data are the result of counting or measuring the attributes of a population. Amount of money, pulse rate, weight, number of people living in a town, and number of students who opt for statistics are examples of quantitative data.
Quantitative data may be either discrete or continuous. All quantitative data that take on only specific numerical...
Quantitative data may be either discrete or continuous. All quantitative data that take on only specific numerical...
28.9K
Classification of Systems-I
191
Linearity is a system property characterized by a direct input-output relationship, combining homogeneity and additivity.
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
191
Classification of Systems-II
150
Continuous-time systems have continuous input and output signals, with time measured continuously. These systems are generally defined by differential or algebraic equations. For instance, in an RC circuit, the relationship between input and output voltage is expressed through a differential equation derived from Ohm's law and the capacitor relation,
150
Aggregates Classification
328
Aggregate classification is generally based on its size, petrographic characteristics, weight, and source. Size classification ranges from coarse to fine aggregates, defined by the size of the particles. Coarse aggregates are particles that do not pass through ASTM sieve No. 4, and aggregates that pass through the sieve are fine aggregates.
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
328


