Human-in-the-Loop Artificial Intelligence: A Systematic Review of Concepts, Methods, and Applications.

Area of Science:

• Artificial Intelligence
• Human-Computer Interaction
• Machine Learning

Background:

• High-stakes applications necessitate human oversight in AI systems where full automation is insufficient.
• Human-in-the-Loop (HITL) AI combines machine learning with human feedback and decision-making.
• Existing research lacks a unified taxonomy and comprehensive review of HITL approaches.

Purpose of the Study:

• To systematically review Human-in-the-Loop AI methodologies, theoretical underpinnings, and applications.
• To propose a novel taxonomy for categorizing HITL systems.
• To identify challenges and outline future research directions for effective human-AI collaboration.

Main Methods:

• Systematic literature review of HITL AI.
• Development of a unified taxonomy based on loop placement, interaction granularity, and temporal characteristics.
• Synthesis of findings across diverse domains including healthcare, autonomous systems, and cybersecurity.

Main Results:

• Categorization of HITL systems using the proposed taxonomy.
• Identification of key challenges such as scalability, cognitive load, and trust calibration.
• Synthesis of domain-specific applications highlighting the essential role of human oversight.

Conclusions:

• HITL AI is crucial for high-risk domains requiring human judgment.
• A structured approach to HITL system design is needed.
• Further research should focus on addressing practical deployment challenges and developing robust human-AI collaboration frameworks.