Related Experiment Video
Updated: Feb 19, 2026
09:47
Author Spotlight: Advancing Alzheimer's Research – Exploring Early Detection and Multi-Omics Approaches
Published on: December 15, 2023
1.9K
Maximum entropy methods for extracting the learned features of deep neural networks
Alex Finnegan1,2, Jun S Song1,2
1Department of Physics, University of Illinois, Urbana-Champaign, Urbana, Illinois, United States of America.
Plos Computational Biology
|October 31, 2017
Summary
We developed a new method to interpret deep neural networks and extract learned features from biological sequences. This approach helps understand what meaningful patterns artificial intelligence models identify in complex data.
Area of Science:
- Computational Biology
- Machine Learning
- Artificial Intelligence
Background:
- Deep neural networks (DNNs) are revolutionizing many scientific fields, but interpreting their learned features remains a significant challenge.
- Understanding the specific biological features DNNs identify is crucial for advancing fields like genomics and molecular biology.
Purpose of the Study:
- To present a general method for interpreting DNNs and extracting learned features from input data.
- To apply this method to biological sequence analysis, specifically identifying transcription factor binding motifs and nucleosome positioning signals.
Main Methods:
- Developed a general algorithm based on statistical physics principles.
- The method samples from a maximum entropy distribution, constrained by the DNN's learned function.
- Applied the framework to analyze biological sequences using data from ChIP-seq and chemical cleavage mapping.
Main Results:
- Successfully identified local transcription factor binding motifs from ChIP-seq data using the developed method.
- Demonstrated that DNNs trained on chemical cleavage nucleosome maps learn nucleosome positioning signals.
- Showed the capability to probe for global sequence features, such as GC content, by imposing additional constraints.
Conclusions:
- The proposed framework provides valuable mathematical tools for interpreting feed-forward neural networks.
- This approach enables the extraction of meaningful biological features learned by DNNs.
- Facilitates a deeper understanding of how AI models process and interpret complex biological sequence data.
Related Concept Videos
01:18Entropy
3.7K
The first law of thermodynamics is quantitatively formulated via an equation relating the internal energy of a system, the heat exchanged by it, and the work done on it. A quantitative formulation of the second law of thermodynamics leads to defining a state function, the entropy.
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
3.7K
02:39Entropy
36.7K
Salt particles that have dissolved in water never spontaneously come back together in solution to reform solid particles. Moreover, a gas that has expanded in a vacuum remains dispersed and never spontaneously reassembles. The unidirectional nature of these phenomena is the result of a thermodynamic state function called entropy (S). Entropy is the measure of the extent to which the energy is dispersed throughout a system, or in other words, it is proportional to the degree of disorder of a...
36.7K
01:20Entropy and the Second Law of Thermodynamics
5.0K
The second law of thermodynamics can be stated quantitatively using the concept of entropy. Entropy is the measure of disorder of the system.
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
5.0K
00:56Extraction: Advanced Methods
1.2K
Metal ions can be separated from one another by complexation with organic ligands–the chelating agent– to form uncharged chelates. Here, the chelating agent must contain hydrophobic groups and behave as a weak acid, losing a proton to bind with the metal. Since most organic ligands used in this process are insoluble or undergo oxidation in the aqueous phase, the chelating agent is initially added to the organic phase and extracted into the aqueous phase. The metal-ligand complex is...
1.2K