Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Deep neural nets as Hamiltonians.

Mike Winer1, Boris Hanin2

  • 1Institute for Advanced Study, University of Maryland, College Park, Maryland 20740, USA.

Physical Review. E
|February 20, 2026
PubMed
Summary
This summary is machine-generated.

This study views random neural networks as Hamiltonians, analyzing their energy landscapes. Infinite-width multilayer perceptrons exhibit complex behaviors, with some nonlinearities showing full replica symmetry breaking.

Related Experiment Videos

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Gibbs Measures from Deep Shaped Multilayer Perceptrons.

Physical review letters·2026
Same author

Sexual identity, sexual behavior, and drug use behaviors among people who use drugs in the rural U.S.

Journal of substance use and addiction treatment·2025
Same author

Bayesian interpolation with deep linear networks.

Proceedings of the National Academy of Sciences of the United States of America·2023
Same author

KSR2 is an essential regulator of AMP kinase, energy expenditure, and insulin sensitivity.

Cell metabolism·2009
Same author

Increased OXPHOS activity precedes rise in glycolytic rate in H-RasV12/E1A transformed fibroblasts that develop a Warburg phenotype.

Molecular cancer·2009
Same journal

Erratum: Low-dimensional model for adaptive networks of spiking neurons [Phys. Rev. E 111, 014422 (2025)].

Physical review. E·2026
Same journal

Disentangling the effects of many-body forces on depletion interactions.

Physical review. E·2026
Same journal

Charge transport and mode transition in dual-energy electron beam diodes.

Physical review. E·2026
Same journal

Optimization of multisite reactions in complex compartmentalized media.

Physical review. E·2026
Same journal

Origin of geometric cohesion in nonconvex granular materials: Interplay between interdigitation and rotational constraints enhancing frictional stability.

Physical review. E·2026
Same journal

Interaction of walkers with a standing Faraday wave.

Physical review. E·2026
See all related articles

Area of Science:

  • Theoretical computer science
  • Statistical mechanics
  • Deep learning theory

Background:

  • Prior work in deep learning theory often analyzes network outputs over random parameters.
  • This study explores the inverse: the energy landscape of a fixed random network over its inputs.

Purpose of the Study:

  • To analyze the energy landscape of a randomly initialized multilayer perceptron (MLP) viewed as a Hamiltonian over its inputs.
  • To investigate the structure of near-global minima in the infinite-width limit.
  • To understand the behavior of different activation functions within this framework.

Main Methods:

  • Viewing a random MLP as a Hamiltonian over inputs.
  • Using the replica trick for exact analytic calculation of entropy (log volume).
  • Deriving and solving saddle-point equations for input overlaps using Gibbs distribution.
  • Numerical solutions for various depths and activation functions (tanh, sin, ReLU, shaped nonlinearities).

Main Results:

  • Exact analytic calculation of entropy at a given energy.
  • Derivation of saddle-point equations describing input overlaps.
  • Numerical solutions reveal diverse behaviors even at infinite width.
  • Full replica symmetry breaking observed for sin activation functions.
  • Replica symmetry observed for shallow tanh/ReLU and deep shaped MLPs.

Conclusions:

  • Random MLPs exhibit rich energy landscape behaviors influenced by activation functions and depth.
  • Infinite-width networks can display complex statistical mechanics phenomena like replica symmetry breaking.
  • The Hamiltonian perspective provides new insights into deep learning theory and the properties of neural network landscapes.