Related Experiment Video
Updated: Jan 11, 2026
11:15
Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
Published on: June 27, 2013
34.3K
Nyström-aware approximations for matrix-based Rényi's entropy
Tieliang Gong1, Wen Wen1, Yuxin Dong1
1School of Computer Science of Technology, Xi'an Jiaotong University, Xi'an, China.
Summary
We developed efficient Nyström-aware approximations for matrix-based Rényi
Area of Science:
- Information Theory
- Machine Learning
- Numerical Analysis
Background:
- Matrix-based Rényi's α-order entropy is valuable for information-theoretical learning.
- Exact computation has prohibitive O(n³) time complexity, limiting its use in large-scale machine learning.
- Existing approximations like Hutch++ require multiple passes over data.
Purpose of the Study:
- To propose novel Nyström-aware approximation strategies for matrix-based Rényi's entropy.
- To reduce computational complexity while maintaining accuracy.
- To improve upon existing approximation methods like Hutch++.
Main Methods:
- Developed Nyström-aware sketching techniques for arbitrary α-order matrix-based Rényi's entropy.
- Reduced time complexity to O(n²s) using s queried randomized vectors.
- Focused on symmetric positive semi-definite matrices.
Main Results:
- Achieved significant reduction in computational complexity compared to exact methods.
- Demonstrated superior efficiency over Hutch++-based approximations with fewer data passes.
- Query complexity shown to be near-optimal with respect to approximation error.
- Enabled potential for parallel computation.
Conclusions:
- Nyström-aware approximations offer a computationally efficient and accurate alternative for matrix-based Rényi's entropy.
- The proposed methods outperform existing techniques in terms of speed and data efficiency.
- These approximations are suitable for large-scale machine learning applications.
Related Concept Videos
01:10Entropy Change in Reversible Processes
3.2K
In the Carnot engine, which achieves the maximum efficiency between two reservoirs of fixed temperatures, the total change in entropy is zero. The observation can be generalized by considering any reversible cyclic process consisting of many Carnot cycles. Thus, it can be stated that the total entropy change of any ideal reversible cycle is zero.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
3.2K
03:00Standard Entropy Change for a Reaction
23.9K
Entropy is a state function, so the standard entropy change for a chemical reaction (ΔS°rxn) can be calculated from the difference in standard entropy between the products and the reactants.
23.9K
00:59Propagation of Uncertainty from Random Error
1.6K
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
1.6K
01:24One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation
1.1K
This lesson introduces two critical methods in pharmacokinetics, the Wagner-Nelson and Loo-Riegelman methods, used for estimating the absorption rate constant (ka) for drugs administered via non-intravenous routes. The Wagner-Nelson method relates ka to the plasma concentration derived from the slope of a semilog percent unabsorbed time plot. However, it is limited to drugs with one-compartment kinetics and can be impacted by factors like gastrointestinal motility or enzymatic degradation.
On...
On...
1.1K
01:29Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
271
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
271
01:20Entropy and the Second Law of Thermodynamics
4.7K
The second law of thermodynamics can be stated quantitatively using the concept of entropy. Entropy is the measure of disorder of the system.
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
4.7K