The effects of presenting imprecise probabilities in intelligence forecasts.

Area of Science:

• Risk and Policy Analysis
• Decision Science
• Cognitive Psychology

Background:

• Assessing and communicating analytic uncertainty to policymakers is crucial for effective risk analysis.
• Forecasting often involves deep uncertainty, making reports complex and prone to ambiguity.

Purpose of the Study:

• To investigate the impact of presenting analytic uncertainty through probability ranges versus point assessments.
• To understand how decision-makers perceive and utilize different communication formats of uncertainty in forecasts.

Main Methods:

• Three studies were conducted using mock intelligence forecasts with narrative evidence and numerical probability assessments.
• Participants evaluated forecasts presented with either probability ranges or point estimates for analytic uncertainty.

Main Results:

• Decision-makers were sensitive to ambiguity communicated via probability ranges.
• Narrative evidence had less impact when paired with probability ranges compared to point assessments.
• Probability ranges were perceived as more useful for high-probability forecasts, while point estimates were favored for low-probability forecasts.
• Hindsight evaluations showed lower blame and higher credibility for forecasts using ranges.

Conclusions:

• Decision-makers may not be inherently ambiguity averse in forecasting contexts.
• Communicating probability and analytic confidence through ranges offers distinct advantages for policymakers.
• Probability ranges can enhance the understanding and utility of uncertain forecasts.