Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Weighted Mean00:57

Weighted Mean

5.3K
While taking the arithmetic, geometric, or harmonic mean of a sample data set, equal importance is assigned to all the data points. However, all the values may not always be equally important in some data sets. An intrinsic bias might make it more important to give more weightage to specific values over others.
For example, consider the number of goals scored in the matches of a tournament. While computing the average number of goals scored in the tournament, it may be more important to...
5.3K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Edge Reconstruction in a Quantum Spin Hall Insulator.

Physical review letters·2026
Same author

Friedel Oscillations in Nanoconfined ^{4}He.

Physical review letters·2026
Same author

Berezinskii-Kosterlitz-Thouless Renormalization Group Flow at a Quantum Phase Transition.

Physical review letters·2025
Same author

Effect of the Soliton Width on Nonequilibrium Exchange Phases of Anyons.

Physical review letters·2024
Same author

Boundary between noise and information applied to filtering neural network weight matrices.

Physical review. E·2023
Same author

Machine Learning Optimization of Majorana Hybrid Nanowires.

Physical review letters·2023

Related Experiment Video

Updated: Aug 16, 2025

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks
11:18

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks

Published on: March 2, 2015

10.4K

Random matrix analysis of deep neural network weight matrices.

Matthias Thamm1, Max Staats1, Bernd Rosenow1

  • 1Institut für Theoretische Physik, Universität Leipzig, Brüderstrasse 16, 04103 Leipzig, Germany.

Physical Review. E
|December 23, 2022
PubMed
Summary

Researchers used random matrix theory to analyze deep neural networks. Most singular values appear random, but those linked to learned information deviate, offering insights into network function and learning regimes.

More Related Videos

Author Spotlight: Advancing Alzheimer's Research – Exploring Early Detection and Multi-Omics Approaches
09:47

Author Spotlight: Advancing Alzheimer's Research – Exploring Early Detection and Multi-Omics Approaches

Published on: December 15, 2023

1.2K
Deep Neural Networks for Image-Based Dietary Assessment
13:19

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

9.3K

Related Experiment Videos

Last Updated: Aug 16, 2025

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks
11:18

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks

Published on: March 2, 2015

10.4K
Author Spotlight: Advancing Alzheimer's Research – Exploring Early Detection and Multi-Omics Approaches
09:47

Author Spotlight: Advancing Alzheimer's Research – Exploring Early Detection and Multi-Omics Approaches

Published on: December 15, 2023

1.2K
Deep Neural Networks for Image-Based Dietary Assessment
13:19

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

9.3K

Area of Science:

  • Machine Learning
  • Statistical Physics
  • Deep Learning Theory

Background:

  • Deep neural networks (DNNs) excel in various applications, necessitating a theoretical understanding of their information storage mechanisms.
  • Analyzing the internal structure of DNNs, specifically weight matrices, is crucial for deciphering their task-specific information encoding.

Purpose of the Study:

  • To investigate the statistical properties of weight matrices in trained DNNs using random matrix theory (RMT).
  • To determine which components of DNNs store learned information and to differentiate between learning regimes.

Main Methods:

  • Application of random matrix theory (RMT) to analyze the singular value decomposition (SVD) of DNN weight matrices.
  • Comparison of singular value statistics and eigenvector entries with universal predictions from RMT and the Porter-Thomas distribution.
  • Utilizing the Hill estimator to analyze the spectral distribution of large singular values.

Main Results:

  • The majority of singular values in DNN weight matrices exhibit statistics consistent with universal RMT predictions, suggesting randomness.
  • Eigenvector entries largely follow the Porter-Thomas distribution, supporting the hypothesis of randomness for most eigenvectors.
  • Deviations from RMT predictions are observed primarily in eigenvectors associated with the largest singular values, indicating potential encoding of learned information.
  • The study successfully distinguishes between different learning regimes (lazy vs. rich) by comparing RMT predictions.
  • The spectral distribution of large singular values generally does not follow a power-law type.

Conclusions:

  • Most of the DNN weight matrix is effectively random, with learned information likely concentrated in a few specific components linked to large singular values.
  • RMT provides a powerful framework for understanding information storage in DNNs and for characterizing different learning dynamics.
  • The findings contribute to a deeper theoretical foundation for deep learning, bridging statistical physics and machine learning.