相关实验视频
Updated: Sep 14, 2025

05:37
An R-Based Landscape Validation of a Competing Risk Model
Published on: September 16, 2022
2.2K
量化AI的不确定性:测试LLM的信任判断的准确性
Trent N Cash1,2, Daniel M Oppenheimer3,4, Sara Christie4
1Department of Social and Decision Sciences, Carnegie Mellon University, 5000 Forbes Ave., 224 Porter Hall, Pittsburgh, PA, 15213, USA. trentncash@gmail.com.
Memory & cognition
|July 22, 2025
概括
大型语言模型 (LLM) 聊天机器人在信任判断方面表现出强大的元认知准确性,与人类相似. 然而,LLM,特别是ChatGPT和Gemini,很难根据过去的表现来调整信心,这揭示了一个关键的局限性.
科学领域:
- 人工智能的人工智能
- 认知科学 认知科学
- 人与计算机的交互
背景情况:
- 像ChatGPT和Gemini这样的大型语言模型 (LLM) 正在改变信息获取.
- 对于人类不确定性量化而言,超认知自信判断至关重要.
- 在LLM信任判断的准确性仍然在很大程度上未被探索.
研究的目的:
- 通过信心判断来调查LLM通过信心判断量化不确定性的能力.
- 为了在各种任务中比较LLM和人类的元认知准确性.
- 为了确定LLM和人类之间的信任判断策略的相似之处和差异.
主要方法:
- 四位LLM (ChatGPT,Bard/Gemini,Sonnet,Haiku) 和人类参与者评估了他们对预测和答案的信心.
- 研究涵盖了随机不确定性 (NFL,奥斯卡预测) 和认识不确定性 (Pictionary,事,大学生活问题).
- 分析了信心判断的绝对和相对准确性.
主要成果:
- 与人类相比,LLM的绝对和相对的元认知准确性是可比的,有时甚至更高.
- 无论是LLM还是人类,都对自己的判断表现出过度自信.
- 与人类不同,LLM,特别是ChatGPT和双子座,往往未能根据先前的表现调整信心.
结论:
- 在超认知自信判断方面,LLM具有显著的能力,接近人类的准确度水平.
- 过度自信是LLM和人类共同的特征.
- 对LLM的一个关键局限性是,与人类不同,他们减少了基于经验动态调整信心的能力.
相关概念视频
Uncertainty: Confidence Intervals
4.8K
The confidence interval is the range of values around the mean that contains the true mean. It is expressed as a probability percentage. The interpretation of a 95% confidence interval, for instance, is that the statistician is 95% confident that the true mean falls within the interval. The upper and lower limits of this range are known as confidence limits. The confidence limits for the true mean are estimated from the sample's mean, the standard deviation, and the statistical factor...
4.8K
Confidence Intervals
7.1K
An unbiased point estimate is often insufficient to predict a population estimate, such as population mean or population proportion. In this scenario, a confidence interval is used. A confidence interval is an estimate similar to a sample proportion. However, unlike the point estimate which is a single value, the confidence interval contains a range of values. These values have lower and upper limits, known as confidence limits, and can be designated as L1 and L2, respectively.
A...
A...
7.1K
Interpretation of Confidence Intervals
6.6K
A confidence interval is a better estimate of the population than a point estimate, as it uses a range of values from a sample instead of a single value.
Confidence intervals have confidence coefficients that are crucial for their interpretation. The most common confidence coefficients are 0.90, 0.95, and 0.99, which can be written as percentages–90%, 95%, and 99%, respectively.
Suppose a person calculates a confidence interval with a confidence coefficient of 0.95. In that case, they can...
Confidence intervals have confidence coefficients that are crucial for their interpretation. The most common confidence coefficients are 0.90, 0.95, and 0.99, which can be written as percentages–90%, 95%, and 99%, respectively.
Suppose a person calculates a confidence interval with a confidence coefficient of 0.95. In that case, they can...
6.6K
Uncertainty in Measurement: Accuracy and Precision
81.5K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.
81.5K
Confidence Coefficient
7.9K
The confidence coefficient is also known as the confidence level or degree of confidence. It is the percent expression for the probability, 1-α, that the confidence interval contains the true population parameter assuming that the confidence interval is obtained after sufficient unbiased sampling; for example, if the CL = 90%, then in 90 out of 100 samples the interval estimate will enclose the true population parameter. Here α is the area under the curve, distributed equally under...
7.9K
Uncertainty: Overview
986
In analytical chemistry, we often perform repetitive measurements to detect and minimize inaccuracies caused by both determinate and indeterminate errors. Despite the cares we take, the presence of random errors means that repeated measurements almost never have exactly the same magnitude. The collective difference between these measurements - observed values - and the estimated or expected value is called uncertainty. Uncertainty is conventionally written after the estimated or expected value.
986

