From human teams to hybrid intelligence teams: identifying, characterizing, and evaluating foundational quality...

Area of Science:

• Computer Science
• Human-Computer Interaction
• Artificial Intelligence

Background:

• Hybrid Intelligence (HI) integrates artificial intelligence (AI) to augment human capabilities.
• Existing literature lacks systematic models for HI system design and evaluation.
• Current discussions often overlook the human-AI ensemble, focusing primarily on technology.

Purpose of the Study:

• To develop a quality model for characterizing and evaluating HI systems from a human-AI team perspective.
• To investigate the applicability of human team properties to HI systems.
• To provide a framework for the holistic design and assessment of HI.

Main Methods:

• Conducted a study with 50 HI researchers to assess the relevance of human team properties for HI.
• Developed a quality model for HI teams with seven high-level and 16 specific quality attributes.
• Performed a second empirical investigation using competitions where participants applied the quality model to HI scenarios.

Main Results:

• Human team properties such as boundedness, interdependence, competency, purposefulness, initiative, normativity, and effectiveness are important for HI systems.
• The developed quality model, comprising seven high-level and 16 specific attributes, was confirmed as relevant through empirical evaluation.
• Analysis of 48 HI usage scenarios highlighted insights gained from applying the quality model during system design.

Conclusions:

• A novel quality model for Hybrid Intelligence teams has been established, grounded in human team properties.
• The model provides a structured approach for designing and evaluating HI systems, emphasizing the human-AI collaboration.
• The findings offer practical guidance for researchers and designers aiming to create effective HI systems.