重新思考开放词汇细分的评估指标
概括
本研究通过引入考虑类别相似性的新指标来解决开放词汇细分评估的局限性. 这些新的指标,Open mIoU,Open AP和Open PQ,改善了对细分模型能力的评估.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 图像处理 图像处理
背景情况:
- 目前的开放词汇细分评估依赖于封闭集指标,忽视预测和基本真相类别之间的语义相似性.
- 这种方法限制了对零射击或跨数据集细分性能的准确评估.
研究的目的:
- 解决现有评估指标在开放词汇细分中的局限性.
- 提出新的评估指标,包括类别之间的语义相似性.
主要方法:
- 通过语言统计,文本嵌入和语言模型,调查了11个单词相似度测量.
- 进行了全面的定量分析和用户研究,以验证相似性测量.
- 设计和实施了三个新的评估指标:开放的mIoU,开放的AP和开放的PQ.
主要成果:
- 拟议的指标,Open mIoU,Open AP和Open PQ,在三个任务中的十二种开放词汇细分方法上进行了基准测试.
- 证明了新型指标有效地评估了细分模型的开放词汇能力,尽管相似度的内在主观性.
- 与传统的封闭式方法相比,这些指标提供了更细致的评估.
结论:
- 开发的评估指标为评估开放词汇细分模型提供了更强大,更准确的方法.
- 这项工作鼓励重新评估当前在开放世界环境中测量细分模型性能的实践.
- 拟议的指标旨在通过提供更好的工具来理解和改进开放词汇细分来推动该领域的发展.
相关概念视频
Aggregates Classification
289
Aggregate classification is generally based on its size, petrographic characteristics, weight, and source. Size classification ranges from coarse to fine aggregates, defined by the size of the particles. Coarse aggregates are particles that do not pass through ASTM sieve No. 4, and aggregates that pass through the sieve are fine aggregates.
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
289
Classification of Systems-I
156
Linearity is a system property characterized by a direct input-output relationship, combining homogeneity and additivity.
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
156
Sensitivity, Specificity, and Predicted Value
135
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
135
Classification of Systems-II
119
Continuous-time systems have continuous input and output signals, with time measured continuously. These systems are generally defined by differential or algebraic equations. For instance, in an RC circuit, the relationship between input and output voltage is expressed through a differential equation derived from Ohm's law and the capacitor relation,
119
Classification of Signals
315
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
315
Extraction: Advanced Methods
390
Metal ions can be separated from one another by complexation with organic ligands–the chelating agent– to form uncharged chelates. Here, the chelating agent must contain hydrophobic groups and behave as a weak acid, losing a proton to bind with the metal. Since most organic ligands used in this process are insoluble or undergo oxidation in the aqueous phase, the chelating agent is initially added to the organic phase and extracted into the aqueous phase. The metal-ligand complex is...
390


