Algorithmic Probability-Guided Machine Learning on Non-Differentiable Spaces.

Area of Science:

• Theoretical Computer Science
• Machine Learning
• Complexity Theory

Background:

• Current machine learning often relies on differentiable programming and large datasets.
• Algorithmic probability and complexity offer a discrete, model-driven perspective.
• Bridging discrete and continuous approaches remains a challenge.

Purpose of the Study:

• To introduce complexity theory into machine learning.
• To demonstrate a model-driven approach requiring less training data and offering greater resilience.
• To bridge discrete computability theory with continuous optimization methods.

Main Methods:

• Utilizing algorithmic complexity and probability to define order in discrete algorithmic spaces.
• Employing a loss function parametrized by algorithmic complexity for regression and classification.
• Developing algorithmically directed search techniques for non-smooth manifolds.

Main Results:

• Machine learning successfully performed on non-smooth surfaces using algorithmic complexity.
• Algorithmic-probability classifiers bridge discrete computability and continuous optimization.
• Developed methods for algorithmic search in discrete, non-differentiable spaces.
• Applications include resilient image classification and parameter identification from noisy data.

Conclusions:

• Complexity theory provides a powerful framework for advancing machine learning.
• Model-driven, discrete approaches can complement continuous methods like deep learning.
• This research opens new avenues for robust and data-efficient machine learning.