Assessment of Situation Awareness Conflict Risk between Human and AI in Process System Operation.

Area of Science:

• Process System Engineering
• Human-AI Interaction
• Process Safety

Background:

• Human-AI conflict is a growing concern in Process System Engineering, encompassing observation, action, and interpretation conflicts.
• Interpretation conflict, linked to situation awareness, is often perceived as confusion or error but lacks sufficient research emphasis.
• Existing studies primarily focus on observation and action conflicts, neglecting the critical aspect of interpretation conflict.

Purpose of the Study:

• To introduce a novel methodology for quantifying the probability and risk of interpretation conflict.
• To address the under-researched area of interpretation conflict in human-AI systems.
• To provide a framework for understanding and managing interpretation conflict in process safety.

Main Methods:

• Development of a new methodology to quantify interpretation conflict probability and risk.
• Demonstration, testing, and validation of the methodology on a two-phase separator system.
• Analysis of how various factors like sensor faults, cyberattacks, and human errors contribute to interpretation conflict.

Main Results:

• Interpretation conflict is often hidden, masked by traditional faults, or obscured by noise in observations and interpretations.
• Sensor faults, logic errors, cyberattacks, human mistakes, and misunderstandings are identified as common triggers for interpretation conflict.
• The proposed methodology successfully quantifies interpretation conflict in a practical process system.

Conclusions:

• Interpretation conflict poses a significant, often underestimated, risk to process safety.
• The developed methodology provides a crucial tool for identifying and assessing interpretation conflict.
• This work lays the foundation for developing effective strategies to manage interpretation conflict and enhance overall system safety.