Algorithm, expert, or both? Evaluating the role of feature selection methods on user preferences and reliance

Area of Science:

• Artificial Intelligence
• Human-Computer Interaction
• Machine Learning

Background:

• User and expert integration in AI development is a key research area.
• AI acceptance as decision support is influenced by various factors.
• Transparency in machine learning models increases the importance of feature selection.

Purpose of the Study:

• Investigate user preferences for expert integration in AI development.
• Analyze how feature selection methods affect user reliance on AI decision support.
• Identify the attitude-behavior gap in AI system usage.

Main Methods:

• Experimental study comparing algorithm-based, expert-based, and combined feature selection.
• Treatment 1: Analyzed user preferences for feature selection methods.
• Treatment 2: Assessed advice reliance by assigning users to different methods.

Main Results:

• Users preferred the combined method, followed by expert-based, then algorithm-based.
• Despite preferences, users showed equal reliance on all methods in actual usage.
• No significant difference in reliance was found when users could choose their preferred method.

Conclusions:

• A significant attitude-behavior gap exists in AI decision support systems.
• Understanding cognitive processes is crucial for effective human-AI interaction.
• Behavioral experiments are essential for evaluating AI system design and user reliance.