Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machines

Area of Science:

• Machine Learning
• Statistical Physics
• Quantum Physics

Background:

• Unsupervised generative modeling aims to learn data distributions.
• Energy-based models (statistical physics) and quantum states (quantum physics) offer distinct approaches.
• Classical and quantum information patterns guide model design.

Purpose of the Study:

• To compare statistical and quantum physics inspired generative models for classical data.
• To analyze information theoretical bounds and inductive biases.
• To evaluate model performance on benchmark datasets.

Main Methods:

• Utilized Restricted Boltzmann Machines (RBM) as a model example.
• Estimated classical mutual information for MNIST datasets.
• Calculated quantum Rényi entropy for Matrix Product States (MPS) representations.
• Compared RBM architectures for learning efficiency.

Main Results:

• Both statistical and quantum approaches exhibited low information complexity patterns.
• Information measures were significantly below theoretical upper bounds.
• RBMs with local sparse connections showed high learning efficiency on MNIST.
• Similar patterns in classical and quantum information measures were observed.

Conclusions:

• A common inductive bias towards low information complexity exists in both approaches.
• Tensor network states show promise for machine learning applications.
• The study highlights the interplay between physics-inspired models and data complexity.