Related Experiment Video
Updated: Jul 7, 2026
11:15
Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
Published on: June 27, 2013
Representation and generalization properties of class-entropy networks.
S Ridella1, S Rovetta, R Zunino
1Department of Biophysical and Electronic Engineering, University of Genoa, 16145 Genova, Italy.
IEEE Transactions on Neural Networks
|February 7, 2008
Summary
Conditional Class Entropy (CCE) networks leverage classification-relevant information for improved data modeling. This approach enhances feedforward network accuracy and generalization through data partitioning and local distribution modeling.
Area of Science:
- Machine Learning
- Artificial Intelligence
- Computer Science
Background:
- Feedforward networks often struggle to fully utilize classification-relevant information.
- Existing cost functions may limit the network's ability to model local data distributions.
Purpose of the Study:
- To introduce and analyze Conditional Class Entropy (CCE) as a novel cost function for feedforward networks.
- To demonstrate CCE's capability in enhancing information exploitation and data space partitioning.
- To investigate the theoretical properties and practical applications of CCE-based networks.
Main Methods:
- Utilizing Conditional Class Entropy (CCE) as the primary cost function.
- Arranging the data space into partitions with unambiguous symbols and class labels.
- Employing a plastic algorithm for network training and region labeling.
- Proposing analytical criteria and practical procedures to improve generalization.
Main Results:
- CCE-based networks effectively model empirical data distributions at a local level.
- Theoretical properties regarding convergence and generalization ability are proven.
- Experimental validation on artificial and real-world datasets confirms network accuracy.
- Proposed methods demonstrably enhance generalization performance.
Conclusions:
- Conditional Class Entropy offers a powerful mechanism for feedforward networks to exploit classification-relevant information.
- CCE-based networks exhibit strong performance in both training convergence and runtime generalization.
- The proposed analytical and practical enhancements further solidify the utility of CCE in machine learning applications.
Related Concept Videos
01:20Entropy and the Second Law of Thermodynamics
The second law of thermodynamics can be stated quantitatively using the concept of entropy. Entropy is the measure of disorder of the system.
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
01:26Entropy and the Second Law of Thermodynamics
Consider an isolated system in which a hot object is placed in contact with a cold one. This is an irreversible process that eventually leads both objects to reach the same equilibrium temperature. It is crucial to note that the constituents of any substance exhibit increased disorder at higher temperatures. As a cold substance absorbs heat, its constituents become more disordered. The energy transfer from a hotter object to a cooler one increases the system's disorder or randomness. This...
02:39Entropy
Salt particles that have dissolved in water never spontaneously come back together in solution to reform solid particles. Moreover, a gas that has expanded in a vacuum remains dispersed and never spontaneously reassembles. The unidirectional nature of these phenomena is the result of a thermodynamic state function called entropy (S). Entropy is the measure of the extent to which the energy is dispersed throughout a system, or in other words, it is proportional to the degree of disorder of a...
01:18Entropy
The first law of thermodynamics is quantitatively formulated via an equation relating the internal energy of a system, the heat exchanged by it, and the work done on it. A quantitative formulation of the second law of thermodynamics leads to defining a state function, the entropy.
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
01:10Entropy Change in Reversible Processes
In the Carnot engine, which achieves the maximum efficiency between two reservoirs of fixed temperatures, the total change in entropy is zero. The observation can be generalized by considering any reversible cyclic process consisting of many Carnot cycles. Thus, it can be stated that the total entropy change of any ideal reversible cycle is zero.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
01:14The Entropy as a State Function
Consider an arbitrary process that moves between two specific states (A and B) in a cyclic manner. This process is reversible and broken down into smaller parts that each follow a Carnot cycle. A Carnot cycle has two isothermal (constant temperature) processes. During these processes, the ratio of the amount of heat transferred to their respective temperature remains constant. The other two processes in the Carnot cycle are also reversible but adiabatic, which means they occur without any heat...