Human control of AI systems: from supervision to teaming

Area of Science:

• Artificial Intelligence
• Human-Computer Interaction
• Cognitive Science

Background:

• AI systems, particularly foundation models, exhibit non-deterministic behavior.
• Ensuring effective AI deployment requires robust human control mechanisms.
• Existing approaches to human control may be inadequate for advanced AI.

Purpose of the Study:

• To review and compare supervisory human control and human-machine teaming for AI systems.
• To assess the adequacy of these approaches for controlling foundation models.
• To propose an improved framework for human control of AI.

Main Methods:

• Literature review of human control approaches in AI.
• Conceptual analysis of supervisory control versus human-machine teaming.
• Examination of case studies from the defense and security domain.

Main Results:

• Supervisory control and human-machine teaming differ in their operational interplay between human and AI behavior.
• Foundation models present unique challenges for traditional control paradigms.
• Practical examples highlight difficulties in human control of AI in defense contexts.

Conclusions:

• Human control of AI is better conceptualized as collaborative agency within a multi-agent system.
• Exclusive human supervision is less effective for complex, non-deterministic AI.
• A shift towards understanding AI control as a joint endeavor is recommended.