Effects of reliability indicators on usage, acceptance and preference of predictive process management decision...

Area of Science:

• Human-Computer Interaction
• Decision Science
• Construction Management

Background:

• Decision Support Systems (DSS) aim to improve user decision-making through predictions and recommendations.
• User trust in DSS remains a challenge despite advancements in data and analytics.
• Trust calibration in DSS is crucial for effective adoption, especially in uncertain environments.

Purpose of the Study:

• To investigate the impact of integrated reliability indication in DSS user interfaces on user trust and decision-making.
• To analyze how reliability information affects user reliance and confidence in DSS during first-time usage.
• To explore user perceptions of DSS usefulness and reliability when presented with explicit reliability cues.

Main Methods:

• A mixed-methods user study involving ten construction industry experts.
• Comparative assessment of two DSS versions: one with and one without reliability indication.
• Scenario-based evaluation in a renovation project context to simulate uncertainty.

Main Results:

• Users stated a need for full information access but showed increased willingness to make preliminary decisions with reliability indicators.
• User confidence and reliance adapted to the indicated reliability levels within the DSS.
• Reliability indication positively influenced subjective usefulness and perceived system reliability.

Conclusions:

• Displaying reliability information in DSS interfaces, using multi-level details and visualizations (e.g., traffic lights with explanations), can foster trust and preliminary decision-making.
• Trustworthy decision support in complex environments requires careful design of reliability indications.
• Further research is needed to optimize trust calibration in advanced DSS for specialized tasks.