Related Experiment Video
Updated: Jul 17, 2026

Dynamic Navigation for Dental Implant Placement
Published on: September 13, 2022
Supporting trust calibration and the effective use of decision aids by presenting dynamic system confidence
John M McGuirl1, Nadine B Sarter
1Department of Industrial and Systems Engineering, Ohio State University, Columbus, USA.
Objective:
To examine whether continually updated information about a system's confidence in its ability to perform assigned tasks improves operators' trust calibration in, and use of, an automated decision support system (DSS).
Background:
The introduction of decision aids often leads to performance breakdowns that are related to automation bias and trust miscalibration. This can be explained, in part, by the fact that operators are informed about overall system reliability only, which makes it impossible for them to decide on a case-by-case basis whether to follow the system's advice.
Method:
The application for this research was a neural net-based decision aid that assists pilots with detecting and handling in-flight icing encounters. A multifactorial experiment was carried out with two groups of 15 instructor pilots each flying a series of 28 approaches in a motion-base simulator. One group was informed about the system's overall reliability only, whereas the other group received updated system confidence information.
Results:
Pilots in the updated group experienced significantly fewer icing-related stalls and were more likely to reverse their initial response to an icing condition when it did not produce desired results. Their estimate of the system's accuracy was more accurate than that of the fixed group.
Conclusion:
The presentation of continually updated system confidence information can improve trust calibration and thus lead to better performance of the human-machine team.
Application:
The findings from this research can inform the design of decision support systems in a variety of event-driven high-tempo domains.
Related Concept Videos
Uncertainty: Confidence Intervals
Interpretation of Confidence Intervals
Confidence intervals have confidence coefficients that are crucial for their interpretation. The most common confidence coefficients are 0.90, 0.95, and 0.99, which can be written as percentages–90%, 95%, and 99%, respectively.
Suppose a person calculates a confidence interval with a confidence coefficient of 0.95. In that case, they can...
Confidence Coefficient
Decision Making: P-value Method
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can have a...
Confidence Intervals
A confidence...
Decision Making: Traditional Method
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...