Predictive models of safety based on audit findings: Part 1: Model development and reliability

Area of Science:

• Aviation safety
• Human Factors/Ergonomics
• Safety management systems

Background:

• Limited research exists on validating safety audit tools against actual safety performance outcomes.
• Aviation maintenance provides a regulated environment with existing audit and safety outcome data.

Purpose of the Study:

• To quantitatively validate safety audit tools as predictors of safety performance.
• To address the gap in studies testing audit validity against safety outcomes.

Main Methods:

• Developed a Human Factors/Ergonomics classification framework based on the HFACS model for classifying human errors found in audits.
• Tested the framework's measurement reliability using four participants classifying 1238 audit reports.
• Achieved Kappa values between 0.5 and 0.8, indicating sufficient reliability for error categorization.

Main Results:

• A novel classification framework for human errors detected by safety audits was established.
• Reliability testing demonstrated consistent categorization of errors across different tiers.
• The developed framework showed sufficient measurement reliability for subsequent prediction validity testing.

Conclusions:

• A Human Factors/Ergonomics classification framework is necessary for safety audits to predict safety performance.
• Counting audit findings alone is insufficient for forecasting future safety.
• The validated framework provides a reliable method for analyzing audit data in aviation maintenance.