Extracting automata from neural networks using active learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep learning models, often neural networks, achieve expert-level performance but are typically "black boxes."
• This lack of transparency limits their application and verification.
• Understanding these models is crucial for trust and further development.

Purpose of the Study:

• To develop an active learning framework for extracting automata from neural network classifiers.
• To enhance the interpretability of deep learning models.
• To improve the testing and verification of neural networks.

Main Methods:

• Utilized Angluin's L* algorithm as the learning component.
• Employed the neural network classifier as an oracle for queries.
• Developed an abstraction interpretation method (value, symbol, word abstractions) for query answering.

Main Results:

• Implemented a prototype and evaluated it on an MNIST classifier.
• Identified optimal abstraction parameters (interval number 2, block size 1x28) for best F1 score.
• Compared extracted Deterministic Finite Automata (DFA) against passive learning methods, showing superior performance on MNIST.

Conclusions:

• The active learning framework successfully extracts interpretable automata from neural networks.
• The proposed abstraction interpretation enhances understanding and verification of black-box models.
• The method offers a promising approach for improving transparency in deep learning.