Automated discovery of algorithms from data.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Scientific Discovery

Background:

• Automating the discovery of scientific and engineering principles requires AI to extract explicit rules from experimental data.
• Current methods struggle due to the immense search space of potential functions.
• Discovering generalizable principles remains a significant challenge in AI research.

Purpose of the Study:

• To introduce a novel machine learning method, deep distilling, for automating the discovery of scientific principles.
• To develop an AI approach that learns from data without extensive function space searches.
• To create human-comprehensible algorithms from neural network parameters.

Main Methods:

• The deep distilling method utilizes symbolic essence neural networks to learn from data.
• Network parameters are losslessly condensed into concise, human-readable computer code.
• The distilled code can incorporate complex structures like loops and nested logic.

Main Results:

• The distilled code is orders-of-magnitude more compact than the original neural network.
• The generated algorithms demonstrate out-of-distribution systematic generalization on arithmetic, vision, and optimization tasks.
• Distilled algorithms successfully solved problems significantly larger and more complex than the training data.

Conclusions:

• Deep distilling enables AI to discover generalizable scientific principles from data.
• The method produces human-comprehensible and highly compact algorithms.
• The discovered algorithms can match or exceed the performance of human-designed algorithms, complementing human expertise.