Learning With Interpretable Structure From Gated RNN.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Deep learning model interpretability is crucial, especially for complex architectures like recurrent neural networks (RNNs).
• The internal workings of gated RNNs remain poorly understood, hindering trust and adoption in critical applications.
• Finite-state automata (FSA) offer a more interpretable mechanism for sequential data processing.

Purpose of the Study:

• To develop methods for learning interpretable finite-state automata (FSA) from recurrent neural networks (RNNs).
• To evaluate the trustworthiness and interpretability of learned FSAs compared to their parent RNNs.
• To analyze the impact of gate count on RNN performance and provide design guidelines.

Main Methods:

• Proposed two novel methods to extract FSA structures from RNNs using distinct clustering techniques.
• Conducted experiments on both artificial and real-world datasets to validate the learned FSAs.
• Analyzed the relationship between the number of gates in an RNN and its overall performance.

Main Results:

• Successfully learned interpretable FSAs from RNNs, revealing semantic aggregated states and transition pathways.
• Demonstrated that the learned FSAs are more trustworthy than the RNNs they were derived from.
• Identified an optimal range for the number of gates in RNNs, suggesting 'less is better' for performance.
• The transition graph of learned FSAs provides insights into RNNs' text classification mechanisms.

Conclusions:

• Learned FSAs can serve as a more interpretable and trustworthy substitute for RNNs in safety-critical applications.
• The findings offer practical guidance for designing more efficient and interpretable RNN architectures.
• The learned FSA structures illuminate the intrinsic decision-making processes within RNNs for tasks like text classification.