Opening the AI Black Box: Distilling Machine-Learned Algorithms into Code.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Program Synthesis

Background:

• Neural networks, while powerful, often function as "black boxes," hindering interpretability.
• Understanding the internal algorithms learned by AI models is crucial for trust and debugging.

Purpose of the Study:

• To present a proof-of-concept method, Mechanistic Interpretability Program Synthesis (MIPS), for converting neural networks into executable code.
• To demonstrate the feasibility of synthesizing programs from the learned algorithms within neural networks.

Main Methods:

• MIPS utilizes automated mechanistic interpretability to analyze neural networks.
• It employs an integer autoencoder to transform recurrent neural networks (RNNs) into finite state machines.
• Symbolic regression (Boolean or integer) is applied to extract the learned algorithm.

Main Results:

• MIPS was tested on 62 algorithmic tasks solvable by RNNs.
• The method successfully synthesized programs for 32 tasks.
• MIPS demonstrated complementary performance to GPT-4, solving 13 tasks that GPT-4 did not.

Conclusions:

• The MIPS method offers a novel approach to program synthesis, distinct from large language models that rely on human training data.
• This technique enhances the interpretability and trustworthiness of machine-learned models.
• Further research is needed to scale MIPS for broader applications.