Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Entropy

Entropy

The first law of thermodynamics is quantitatively formulated via an equation relating the internal energy of a system, the heat exchanged by it, and the work done on it. A quantitative formulation of the second law of thermodynamics leads to defining a state function, the entropy.
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...

Introduction to Learning

Introduction to Learning

Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...

Entropy Change in Reversible Processes

Entropy Change in Reversible Processes

In the Carnot engine, which achieves the maximum efficiency between two reservoirs of fixed temperatures, the total change in entropy is zero. The observation can be generalized by considering any reversible cyclic process consisting of many Carnot cycles. Thus, it can be stated that the total entropy change of any ideal reversible cycle is zero.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.

Associative Learning

Associative Learning

Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...

Observational Learning

Observational Learning

Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...

Propagation of Uncertainty from Random Error

Propagation of Uncertainty from Random Error

An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Learning in Convolutional Neural Networks Accelerated by Transfer Entropy.

Entropy (Basel, Switzerland)·2021

Same author

Semiotic Aggregation in Deep Learning.

Entropy (Basel, Switzerland)·2020

Same author

Transfer Information Energy: A Quantitative Indicator of Information Transfer between Time Series.

Entropy (Basel, Switzerland)·2020

See all related articles

Search research articles

Related Experiment Video

Updated: Nov 27, 2025

Decoding Natural Behavior from Neuroethological Embedding

Decoding Natural Behavior from Neuroethological Embedding

Published on: October 3, 2025

Learning in Feedforward Neural Networks Accelerated by Transfer Entropy.

Adrian Moldovan^1,2, Angel Caţaron^1,2, Răzvan Andonie³

¹Department of Electronics and Computers, Transilvania University, 500024 Braşov, Romania.

Entropy (Basel, Switzerland)

|December 8, 2020

Summary

This summary is machine-generated.

This study introduces an information-theoretical method using transfer entropy (TE) to analyze neural network information flow. A novel training algorithm leverages TE feedback connections for improved efficiency in complex neural networks.

Keywords:

backpropagation causality deep learning gradient descent neural network transfer entropy

Related Experiment Videos

Last Updated: Nov 27, 2025

Decoding Natural Behavior from Neuroethological Embedding

Decoding Natural Behavior from Neuroethological Embedding

Published on: October 3, 2025

Area of Science:

Artificial Intelligence
Machine Learning
Information Theory

Background:

Neural network training is increasingly challenging due to large, complex datasets.
Existing methods lack efficient ways to leverage causal relationships within networks.

Purpose of the Study:

To design more efficient neural network training algorithms.
To utilize causal relationships inferred from neural networks via information theory.

Main Methods:

Developed an information-theoretical method to analyze information transfer between feedforward neural network nodes.
Measured information transfer using transfer entropy (TE) of feedback connections.
Introduced a backpropagation-based training algorithm incorporating TE feedback.

Main Results:

Demonstrated a method to quantify information transfer relevance in neural network connections.
Showcased improved training performance by utilizing TE feedback connections.

Conclusions:

Transfer entropy offers a viable approach to analyze and enhance neural network training.
The proposed method provides a novel way to improve the efficiency of training complex neural networks.