人群的智慧:LLM整体预测能力与人类人群准确度相美
Philipp Schoenegger1, Indre Tuminauskaite2, Peter S Park3
1Department of Management, London School of Economics and Political Science, Houghton Street, London WC2A 2AE, UK.
Science advances
|November 8, 2024
概括
聚合的大型语言模型 (LLM) 可以与人类人群预测准确度相匹配. 结合人类和LLM预测进一步提高了准确性,表明了提高集体智能的潜力.
科学领域:
- 人工智能的人工智能
- 认知科学 认知科学
- 计算社会科学 计算社会科学
背景情况:
- "群众智慧"效应通过聚合提高了人类预测的准确性.
- 个别的大型语言模型 (LLM) 在历史上表现不佳,与人类总体预测相比.
研究的目的:
- 调查一组LLM是否可以模拟"群众智慧"效应.
- 为了比较LLM人群与人类人群的预测准确度.
- 通过结合人类预测来探索改善LLM预测的方法.
主要方法:
- 一组12个LLM生成了31个二进制问题的概率预测.
- 在3个月的时间里,LLM预测与925个人类预测者进行了比较.
- 在暴露于人类中位数预测后,LLM预测也被评估 (GPT-4,Claude 2).
主要成果:
- 总的来说,LLM人群的表现在统计学上是无法与人类人群区分的.
- 士学位预测显示了类似人类的偏见,包括默许偏见.
- 当接触到人类预测中位数时,LLM的准确性提高了17-28%,简单的平均值产生了最佳结果.
结论:
- 聚合的LLM预测可以与人类人群的预测准确度相美.
- 在预测任务中,LLM表现出类似人类的偏见.
- 结合人类和LLM预测,为增强集体智能和预测准确性提供了一个有希望的途径.
相关概念视频
Prediction Intervals
2.2K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.2K
Predicting Reaction Outcomes
8.2K
Kinetics describes the rate and path by which a reaction occurs. In contrast, thermodynamics deals with state functions and describes the properties, behavior, and components of a system. It is not concerned with the path taken by the process and cannot address the rate at which a reaction occurs. Although it does provide information about what can happen during a reaction process, it does not describe the detailed steps of what appears on an atomic or a molecular level. On the other hand,...
8.2K
End Point Prediction: Gran Plot
280
A Gran plot is used to predict the equivalence volume or endpoint of a potentiometric or acid-base titration without reaching the endpoint. Typically, titration data is collected as a function of the titrant's volume up to a point less than the equivalence volume and then transformed into a linear format. The straight line is extended to the x-axis, indicating the necessary titrant volume to achieve the equivalence point.
For potentiometric titration, the Gran plot is created by plotting...
For potentiometric titration, the Gran plot is created by plotting...
280
Sensitivity, Specificity, and Predicted Value
194
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
194
Accuracy and Precision
8.7K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value. Highly accurate...
8.7K
Social Facilitation
31.8K
Not all intergroup interactions lead to negative outcomes. Sometimes, being in a group situation can improve performance. Social facilitation occurs when an individual performs better when an audience is watching than when the individual performs the behavior alone. This typically occurs when people are performing a task for which they are skilled.
31.8K


