Complete and partial fault tolerance of feedforward neural nets

Area of Science:

• Artificial Intelligence
• Computer Engineering
• Network Science

Background:

• Artificial neural networks (ANNs) are susceptible to permanent faults.
• Estimating and improving fault tolerance (FT) in ANNs is crucial for reliable deployment.
• Existing fault models often simplify complex failure modes.

Purpose of the Study:

• To propose a method for estimating the fault tolerance of feedforward ANNs.
• To develop a procedure for synthesizing robust ANNs with enhanced fault tolerance.
• To quantify FT as a function of redundancy and derive analytical bounds.

Main Methods:

• A fault model abstracting permanent stuck-at type faults was used.
• A procedure involving replication of hidden units was developed to build FT ANNs.
• Metrics were devised to quantify FT, and an analytical lower bound for redundancy was derived.

Main Results:

• Less than triple modular redundancy (TMR) cannot guarantee complete FT against all single faults.
• The actual redundancy for complete FT is problem-specific and often higher than the general lower bound.
• Random testing of links provides accurate FT estimates for large ANNs, improving with net size.

Conclusions:

• The conventional TMR scheme is generally optimal for achieving complete FT in most ANNs.
• ANNs exhibit inherent partial FT and degrade gracefully with faults.
• The first replication of hidden units provides the most significant enhancement in partial FT.