对于回归任务中的不确定性定量化的剩余贝叶斯注意力网络
Youliang Chen1,2, Wencan Guan3,4,5, Rafig Azzam6
1Department of Civil Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, 516 Jungong Rd, PR China.
Scientific reports
|November 2, 2025
概括
剩余贝叶斯注意力 (RBA) 框架通过整合贝叶斯推理和变换器来增强深度序列建模中的不确定性量化. 它提供稳定的性能和改进的预测间隔校准,特别是对于结构化数据.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 现代序列建模需要强大的不确定性量化.
- 现有的贝叶斯推理和变压器集成面临着工程挑战.
- 关键问题包括注意力概率化,在剩余连接中不确定性传播,以及解认识论-aleatoric不确定性.
研究的目的:
- 为端到端的概率推理提出剩余贝叶斯注意力 (RBA) 框架.
- 在将贝叶斯式方法与变压器架构集成方面解决系统工程挑战.
- 为深度序列建模提供原则性不确定性量化.
主要方法:
- 开发了贝叶斯式前层,用于可微分参数级不确定性传播.
- 嵌入式辐射基函数内核和适应性Beta分布式权重在多层剩余贝叶斯关注.
- 使用贝叶斯共变量构造与外部积和固有值校正用于严格的共变量表示.
主要成果:
- 瑞银在六个领域的基准数据集上展示了稳定的不确定性量化.
- 在结构化数据的预测间隔校准质量方面取得了技术优势.
- 确定了当前深度学习在多物理合系统建模中的技术局限性.
结论:
- RBA为贝叶斯推理和变压器集成提供了一个系统的工程框架.
- 为深度序列建模中原则性不确定性量化提供了方法论性贡献.
- 突出了适用性边界和对未来研究方向的经验见解.
相关概念视频
Uncertainty: Overview
1.5K
In analytical chemistry, we often perform repetitive measurements to detect and minimize inaccuracies caused by both determinate and indeterminate errors. Despite the cares we take, the presence of random errors means that repeated measurements almost never have exactly the same magnitude. The collective difference between these measurements - observed values - and the estimated or expected value is called uncertainty. Uncertainty is conventionally written after the estimated or expected value.
1.5K
Uncertainty: Confidence Intervals
10.1K
The confidence interval is the range of values around the mean that contains the true mean. It is expressed as a probability percentage. The interpretation of a 95% confidence interval, for instance, is that the statistician is 95% confident that the true mean falls within the interval. The upper and lower limits of this range are known as confidence limits. The confidence limits for the true mean are estimated from the sample's mean, the standard deviation, and the statistical factor...
10.1K
Propagation of Uncertainty from Random Error
1.6K
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
1.6K
Propagation of Uncertainty from Systematic Error
1.3K
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
1.3K
Multi-input and Multi-variable systems
383
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
In the absence of...
383
Uncertainty in Measurement: Accuracy and Precision
99.6K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.
99.6K

