Incentives, expertise, and medical decisions: testing the robustness of natural frequency framing

The natural frequency effect (NF effect)-whereby framing health risks using information presented as natural frequencies (NFs), instead of conditional probabilities (CPs), results in improved diagnostic problem solving-has led to the recommended use of NFs in clinical practice. This experiment tests, via incentivization of a lab-based decision, the hypothesis that the NF effect reflects differential motivation applied to solving problems that differ in complexity. The study examines if incentive effects are moderated by task complexity and expertise and also explores the extent to which NF frames improve diagnostic understanding.

Three-hundred and 25 participants (235 novices and 90 medical experts) were randomly allocated to a frame (NF vs. CP) by task difficulty (short vs. standard menus) by incentive (present vs. absent) between-subjects design. The task was to calculate the positive predictive value (PPV) of the hemoccult test for colorectal cancer. Effort, self-efficacy, and diagnostic understanding were assessed.

Incentives increased correct problem solving and the NF effect was replicated. The NF effect did not vary as a function of incentivization, but was slightly attenuated by task complexity and expertise. There was no evidence that effort mediated the incentive effect. The correct PPV (which is low) was associated with reduced trust in the test's diagnostic accuracy. For those who committed errors, NF frames increased the likelihood of underestimating the PPV, with underestimation associated with greater trust in the test.

The NF effect is robust to incentives supporting the use of NF frames in clinical settings. When errors occur, however, NF frames are linked to underestimation.