Technology-supported self-triage decision making

Area of Science:

• Human-Computer Interaction
• Health Informatics
• Decision Science

Background:

• Laypeople increasingly use Symptom-Assessment Applications (SAAs) and Large Language Models (LLMs) for healthcare navigation.
• Previous research often evaluates human and AI performance separately, neglecting hybrid decision-making dynamics.
• The impact of SAAs and LLMs on laypeople's healthcare decision accuracy remains unclear.

Purpose of the Study:

• To investigate decision-making processes in human-technology teams for healthcare self-triage.
• To assess whether SAAs and LLMs improve laypeople's healthcare decision accuracy.
• To develop a model for technology-assisted self-triage decision-making.

Main Methods:

• Convergent parallel mixed-methods study combining semi-structured interviews and a randomized controlled trial.
• Qualitative interviews explored factors influencing decision-making in human-technology interactions.
• Quantitative analysis compared decision accuracy before and after using an SAA versus an LLM.

Main Results:

• Interview data revealed decision-making is influenced by pre-interaction, during-interaction, and post-interaction factors.
• Users leverage technology for information gathering/analysis but retain responsibility for integration and final decisions.
• Laypeople's decision accuracy significantly improved with a high-performing SAA (53.2% to 64.5%), but not with an LLM (54.8% to 54.2%).

Conclusions:

• SAAs can enhance laypeople's healthcare decision accuracy, supporting technology-assisted self-triage.
• LLMs, in their current form tested, did not demonstrate an improvement in decision accuracy for this context.
• Future research should prioritize studying AI tools within human-in-the-loop frameworks to understand their real-world impact.