Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Transformers with Off-Nominal Turns Ratios01:25

Transformers with Off-Nominal Turns Ratios

213
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
213
Types Of Transformers01:16

Types Of Transformers

1.1K
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
1.1K
Transformers01:26

Transformers

1.2K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.2K
The Ideal Transformer01:26

The Ideal Transformer

926
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
926
Transformers in Distribution System01:27

Transformers in Distribution System

167
Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
167
Three-Winding Transformers01:19

Three-Winding Transformers

315
Three identical single-phase transformers can be configured to form a three-phase transformer connection, which involves high-voltage and low-voltage windings. The high-voltage windings are denoted by capital letters A-B-C, while the low-voltage windings are labeled with lowercase letters a-b-c, representing their respective phases. This notation helps distinguish between the high and low voltage sides of the transformer.
In the per-unit equivalent circuit of a grounded Y-Y three-phase...
315

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Adaptive Signal Processing and Machine Learning Using Entropy and Information Theory.

Entropy (Basel, Switzerland)·2023
Same author

An Overview of Variational Autoencoders for Source Separation, Finance, and Bio-Signal Applications.

Entropy (Basel, Switzerland)·2022
Same author

A Flexible Hybrid BCH Decoder for Modern NAND Flash Memories Using General Purpose Graphical Processing Units (GPGPUs).

Micromachines·2019
See all related articles

Related Experiment Video

Updated: Sep 18, 2025

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images
04:23

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images

Published on: April 21, 2023

2.0K

Information-Theoretical Analysis of a Transformer-Based Generative AI Model.

Manas Deb1, Tokunbo Ogunfunmi1

  • 1Department of Electrical and Computer Engineering, Santa Clara University, Santa Clara, CA 95053, USA.

Entropy (Basel, Switzerland)
|June 26, 2025
PubMed
Summary

This study uses Information Theory to analyze Large Language Models (LLMs). We visualize how Transformers encode word relationships, offering deeper insights than attention scores and aiding in troubleshooting learning issues.

Keywords:
fisher metricgenerative AIgeodesicinformation geometryinformation theorymachine learningmutual information estimationriemann manifoldtransformer

More Related Videos

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

693

Related Experiment Videos

Last Updated: Sep 18, 2025

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images
04:23

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images

Published on: April 21, 2023

2.0K
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

693

Area of Science:

  • Artificial Intelligence
  • Computational Linguistics
  • Information Theory

Background:

  • Large Language Models (LLMs) demonstrate advanced natural language capabilities.
  • Understanding the internal workings of Transformer architectures in LLMs remains a challenge.
  • Generative AI models rely on neural networks and the Transformer architecture for content creation.

Purpose of the Study:

  • To analyze the internal mechanisms of the Transformer architecture using Information Theory.
  • To quantify information flow through Transformer layers by treating them as information transmission channels.
  • To visualize the encoding of word relationships within Transformer layers.

Main Methods:

  • Applying Information Theory to analyze Transformer internals.
  • Computing channel capacity to quantify information transmission.
  • Developing Information-Theoretic tools for visualization on an Information plane.
  • Utilizing Information Geometry to analyze high-dimensional word vectors and geodesic distances on Riemannian manifolds.

Main Results:

  • Developed novel visualizations of word relationship encoding in Transformers.
  • Demonstrated that Information-Theoretic tools provide more insight than attention scores.
  • Showcased the utility of Information Theory in identifying and troubleshooting Transformer learning problems.
  • Quantified information flow through Transformer layers.

Conclusions:

  • Information Theory offers powerful tools for understanding Transformer architectures.
  • Visualizations based on Information Theory reveal deeper insights into word encoding.
  • This approach aids in diagnosing and resolving learning challenges in LLMs.
  • The study enhances interpretability of Generative AI models.