Aggregating predictions from experts: a review of statistical methods, experiments, and applications

Area of Science:

• Decision Science
• Forecasting
• Artificial Intelligence

Background:

• Statistical forecasting models require substantial data, limiting accuracy with sparse or dynamic datasets.
• Expert judgmental forecasts leverage human intuition to predict outcomes with limited data.
• Numerous algorithms exist for aggregating expert predictions, yet no consensus on an optimal method has been reached.

Purpose of the Study:

• To review recent literature on aggregating expert-elicited predictions.
• To identify common terminology, aggregation techniques, and performance metrics in the field.
• To provide guidance for future research in expert forecasting.

Main Methods:

• Systematic literature review of recent research on expert forecasting aggregation.
• Analysis of common terminology, aggregation algorithms, and performance evaluation metrics.
• Synthesis of findings to identify trends and gaps in the research.

Main Results:

• A wide array of aggregation methods for expert predictions have been proposed.
• Performance metrics used to evaluate these methods vary significantly.
• There is a lack of consensus on the most effective aggregation strategy.

Conclusions:

• The field of expert judgmental forecasting is rapidly expanding.
• Standardization of terminology and methodology is needed to advance research.
• Future work should focus on developing and validating optimal aggregation models.