Thermodynamics energy for both supervised and unsupervised learning neural nets at a constant temperature
Related Experiment Videos
1Naval Surface Warfare Center Dahlgren Division, B Dept., VA 22448, USA.
Abstract:
Unified Lyaponov function is given for the first time to prove the learning methodologies convergence of artificial neural network (ANN), both supervised and unsupervised, from the viewpoint of the minimization of the Helmholtz free energy at the constant temperature. Early in 1982, Hopfield has proven the supervised learning by the energy minimization principle. Recently in 1996, Bell & Sejnowski has algorithmically demonstrated Independent Component Analyses (ICA) generalizing the Principal Component Analyses (PCA) that the continuing reduction of early vision redundancy happens towards the "sparse edge maps" by maximization of the ANN output entropy. We explore the combination of both as Lyaponov function of which the proven convergence gives both learning methodologies. The unification is possible because of the thermodynamics Helmholtz free energy at a constant temperature. The blind de-mixing condition for more than two objects using two sensor measurement. We design two smart cameras with short term working memory to do better image de-mixing of more than two objects. We consider channel communication application that we can efficiently mix four images using matrices [AO] and [Al] to send through two channels.
Related Concept Videos
Entropy and the Second Law of Thermodynamics
Entropy and the Second Law of Thermodynamics
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
The First Law of Thermodynamics
Effects of Temperature on Free Energy
Temperature and Thermal Equilibrium
The concept of temperature has evolved from the common concepts of hot and cold. The scientific definition of temperature explains more than just our sense of hot and cold. Temperature is operationally defined as the quantity measured with a thermometer. Furthermore, temperature is...
The Carnot Cycle
What could be the theoretical limit to the efficiency of a heat engine? The...