通过使用机器学习的二进制分类来评估小型数据集的精细方法
Steffen Steinert1,2, Verena Ruf1, David Dzsotjan1
1Chair of Physics Education, Ludwig-Maximilians-Universität München (LMU Munich), Munich, Germany.
PloS one
|May 21, 2024
概括
在教育研究中,对小型数据集的机器学习分析需要仔细评估. 本研究引入了一种精细的方法,使用排列测试和嵌套交叉验证来确保对二进制分类任务的可靠,公正的结果.
科学领域:
- 机器学习 机器学习
- 统计分析 统计分析
- 教育研究教育研究
背景情况:
- 经典的统计方法经常被机器学习 (ML) 补充或取代.
- 在教育研究等领域常见的小型数据集,带来了与偏见和虚假发现有关的挑战.
- 在有限的数据上评估ML性能需要专门的技术来确保可靠性.
研究的目的:
- 在小型数据集上使用ML评估二进制分类性能的精细方法.
- 在数据有限的研究背景下,解决在ML模型评估中的偏见和机会问题.
- 为小数据集ML应用程序选择适当的评估指标提供准则.
主要方法:
- 实施非参数变换测试,以评估ML模型结果的概括性.
- 使用重复嵌套交叉验证来实现无偏差和可靠的性能估计.
- 对各种评估指标进行比较分析,包括马修斯相关系数.
主要成果:
- 重复的嵌套交叉验证显示了最小的偏差和高可靠性,结果在很大程度上独立于机会.
- 顺序测试有效量化了结果概括到新的,未见过的数据的概率.
- 马修斯相关系数被认为是对二进制分类的强大指标,当类具有同等重要性时,显示出低偏差和偶然成功的机会.
结论:
- 建议使用评估指标的组合来培训和评估ML分类器,以利用各自的优势.
- 拟议的方法,包括顺序测试和嵌套交叉验证,对于对小型数据集进行准确的ML分析至关重要.
- 在将机器学习技术应用于小数据集时,避免偏见至关重要,特别是在教育等敏感研究领域.
相关概念视频
Classification of Systems-I
180
Linearity is a system property characterized by a direct input-output relationship, combining homogeneity and additivity.
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
180
Classification of Systems-II
140
Continuous-time systems have continuous input and output signals, with time measured continuously. These systems are generally defined by differential or algebraic equations. For instance, in an RC circuit, the relationship between input and output voltage is expressed through a differential equation derived from Ohm's law and the capacitor relation,
140
Aggregates Classification
317
Aggregate classification is generally based on its size, petrographic characteristics, weight, and source. Size classification ranges from coarse to fine aggregates, defined by the size of the particles. Coarse aggregates are particles that do not pass through ASTM sieve No. 4, and aggregates that pass through the sieve are fine aggregates.
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
317
How Data are Classified: Categorical Data
32.5K
A variable, usually notated by capital letters such as X and Y, is a characteristic or measurement that can be determined for each member of a population. Data are the actual values of variables. They may be numbers, or they may be words. Datum is a single value.
Data are classified based on whether they are measurable or not. Categorical data cannot be measured; instead, it can be divided into categories. For example, if Y denotes a person's party affiliation, some examples of Y include...
Data are classified based on whether they are measurable or not. Categorical data cannot be measured; instead, it can be divided into categories. For example, if Y denotes a person's party affiliation, some examples of Y include...
32.5K
Classification of Signals
445
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
445
Classification of Leukocytes
1.9K
Leukocytes are classified into two groups based on the presence or absence of cytoplasmic granules. Granular leukocytes, which contain granules, belong to the myeloid lineage and are divided into three subtypes: neutrophils, eosinophils, and basophils. These cells are roughly spherical and characterized by the granules in their cytoplasm.
Neutrophils are the most abundant type of granular leukocytes, comprising 50-70% of all leukocytes. They feature small, evenly distributed granules and a...
Neutrophils are the most abundant type of granular leukocytes, comprising 50-70% of all leukocytes. They feature small, evenly distributed granules and a...
1.9K


