Related Experiment Video
Updated: Jul 12, 2026
11:15
Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
Published on: June 27, 2013
Multivariate Extension of Matrix-Based Rényi's α-Order Entropy Functional
IEEE Transactions on Pattern Analysis and Machine Intelligence
|August 10, 2019
Summary
This study introduces a new matrix-based Rényi
Area of Science:
- Information Theory
- Machine Learning
- Data Science
Background:
- Matrix-based Rényi's α-order entropy is defined using RKHS eigenspectra.
- Current theory is limited to single variables or pairwise mutual information.
- Multivariate information quantities are crucial in information theory and machine learning.
Purpose of the Study:
- Define matrix-based Rényi's α-order joint entropy for multiple variables.
- Facilitate estimation of multivariate information quantities.
- Apply the new definition to feature selection and hyperspectral image analysis.
Main Methods:
- Extension of matrix-based Rényi's α-order entropy to multivariate settings.
- Utilizing normalized eigenspectrum of projected data in RKHS.
- Application in feature selection and hyperspectral image band selection.
Main Results:
- Successfully defined matrix-based Rényi's α-order joint entropy.
- Demonstrated ease of estimating interactive information and total correlation.
- Showcased practical utility in feature selection and HSI band selection.
Conclusions:
- The new definition simplifies estimation of complex multivariate information.
- Provides a powerful tool for feature selection and data analysis.
- Successfully applied to a real-world hyperspectral imaging problem.
More Related Videos
Related Concept Videos
01:10Entropy Change in Reversible Processes
In the Carnot engine, which achieves the maximum efficiency between two reservoirs of fixed temperatures, the total change in entropy is zero. The observation can be generalized by considering any reversible cyclic process consisting of many Carnot cycles. Thus, it can be stated that the total entropy change of any ideal reversible cycle is zero.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
01:20Entropy and the Second Law of Thermodynamics
The second law of thermodynamics can be stated quantitatively using the concept of entropy. Entropy is the measure of disorder of the system.
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
01:14The Entropy as a State Function
Consider an arbitrary process that moves between two specific states (A and B) in a cyclic manner. This process is reversible and broken down into smaller parts that each follow a Carnot cycle. A Carnot cycle has two isothermal (constant temperature) processes. During these processes, the ratio of the amount of heat transferred to their respective temperature remains constant. The other two processes in the Carnot cycle are also reversible but adiabatic, which means they occur without any heat...
01:27Applications of Integration to Probability Density Functions
Continuous probability distributions are used to model random variables that can take on any real value within a specified range. These variables do not take on isolated or countable values but rather exist on a continuum. For example, the height of an individual can be measured with increasing precision—such as 163.5 or 165.25 centimeters—demonstrating that height is a continuous random variable.The behavior of such variables is described using a probability density function (PDF), which...
01:25Limits of Multivariable Functions
Limits of multivariable functions describe how a function behaves as its input approaches a particular point in the plane. In single-variable calculus, a limit examines the behavior of a function as the input approaches a number from two directions along a line. For functions of two variables, the situation is more complex because the input can approach a point from infinitely many paths in the xy-plane. A limit exists only when the function approaches the same value along every possible...
01:30Multivariable Functions and Higher Derivatives
A multivariable function assigns a single output value to each ordered set of independent inputs, thereby defining a surface in three-dimensional space. For a function f(x, y), each point (x, y) corresponds to a height z = f(x, y). This geometric interpretation allows systematic analysis of how the output varies as multiple variables change simultaneously. Such functions frequently arise in physical models and optimization problems, where system behavior depends on several interacting...