Equal to or better than: The application of statistical non-inferiority to fatigue risk management

In December 2014, the Federal Aviation Administration (FAA) completed a major revision of the rules and regulations governing flight and duty time in commercial aviation (Federal Aviation Regulation (FAR) Part 117). Scientists were included in the revision process and provided insights into sleep, sleep loss, the circadian rhythm, and their effects on performance that were incorporated into the new rule. If a planned flight was non-compliant with the regulation, for example if it exceeded flight and duty time limits, it could only be flown under an FAA-approved Fatigue Risk Management System (FRMS) as meeting an Alternative Method of Compliance (AMOC). One method that a flight could qualify as an AMOC is if it could be demonstrated empirically that it was as safe as or safer than a similar flight, designated the Safety Standard Operation (SSO), that was compliant with the regulation. In the present paper, we demonstrate the FRMS process using a comparison between a non-compliant AMOC flight from the US west coast to Australia and a compliant SSO flight from the US west coast to Taiwan. The AMOC was non-compliant because it exceeded the flight time limits in the prescriptive rule. Once a data collection exemption was granted by the FAA, both the outbound and inbound AMOC and SSO routes were studied on four Safety Performance Indicators (SPIs). The SPIs studied were inflight sleep, cognitive performance, self-reported fatigue, and self-reported sleepiness. These measures were made at top of descent (TOD), a critical phase of flight. The study was designed as a paired comparison. Forty volunteer pilots studied flew both the AMOC and the SSO flights for a total of 80 studied flights. Using statistical non-inferiority applied to the AMOC and SSO SPIs, we demonstrated, as required by the new rule, that the US-Australia AMOC flight was "as safe as, or safer than" the US-Taiwan SSO flight. In the context of FRMS, statistical non-inferiority is a concept and technique of great utility, straightforward in application, producing clear visual representations of the findings, and providing a direct answer to the question posed by the regulation - is the AMOC flight "as safe as, or safer than" the SSO.