Management by consent in human-machine systems: when and why it breaks down

Area of Science:

• Human-Computer Interaction
• Aviation Psychology
• Automation Safety

Background:

• Operator consent is crucial for human-machine coordination in automated systems.
• Effective management of automated systems requires operators to make informed decisions about machine goals and actions.
• Understanding limitations in operator detection of system conflicts is vital for aviation safety.

Purpose of the Study:

• To investigate how conflict type, time pressure, and display design affect pilot decision-making in a management-by-consent context.
• To identify factors contributing to pilots' failure to detect and intervene in automated system conflicts.
• To inform the design of future aviation automation and digital communication systems.

Main Methods:

• Thirty B757 pilots flew eight descent scenarios with varying air traffic control clearance conflicts.
• Scenarios involved conflicts from incompatible goals or inappropriate automated flight deck system implementation.
• Performance data and verbal protocols were analyzed to understand pilot decision-making processes.

Main Results:

• Pilots frequently failed to detect conflicts, particularly those related to clearance implementation.
• Time pressure significantly impaired conflict detection and intervention capabilities.
• Poor system feedback and difficulties in predicting automation behavior were key explanatory factors.

Conclusions:

• Current automation management strategies may be insufficient for complex aviation environments.
• Future digital air-ground communication systems require careful design to mitigate risks of undetected conflicts.
• Improving system feedback and pilot expectation management is critical for safe human-machine coordination.