Related Experiment Video
Updated: Aug 10, 2026
06:02
Topographical Estimation of Visual Population Receptive Fields by fMRI
Published on: February 3, 2015
Fisher-Rao guided channel pruning with progressive re-estimation
Xinjian Xiang1, Mingjun Lin1, Yongping Zheng1
1Zhejiang University of Science and Technology, Hangzhou, 310023, Zhejiang, China.
Summary
This study introduces a Fisher-Rao guided framework for structured channel pruning, significantly reducing model parameters and FLOPs while maintaining high accuracy. The method offers a reliable and efficient approach to model compression for deep learning applications.
Area of Science:
- Computer Science
- Artificial Intelligence
- Machine Learning
Background:
- Structured channel pruning is key for efficient deep learning models.
- Reliability hinges on importance criteria and pruning schedules.
Purpose of the Study:
- To develop a robust framework for structured channel pruning.
- To improve model compression efficiency and reliability.
Main Methods:
- Proposed a Fisher-Rao guided framework for channel scoring.
- Utilized diagonal empirical Fisher approximation.
- Incorporated progressive re-estimation and short recovery training.
Main Results:
- Achieved significant parameter/FLOP reduction on CIFAR-10 (e.g., ResNet-110: 85.5%/66.8%) and ImageNet (e.g., ResNet-50: 60.0%/50.1%).
- Maintained high accuracy on ImageNet (e.g., ResNet-50: 75.65% Top-1).
- Demonstrated backbone-dependent sensitivity in recovery-free diagnostics.
Conclusions:
- The Fisher-Rao guided framework is an effective baseline for classification pruning.
- The method is loss-coupled and budget-controlled, adaptable to various architectures.
- It offers a reliable approach to deployment-friendly model compression.
Related Concept Videos
01:25Maximizing the Directional Derivative
The directional derivative is a central concept in multivariable calculus that describes how a function changes at a given point when moving in a specified direction. This direction is represented by a unit vector, ensuring that only the orientation influences the rate of change. By varying the direction, different rates of change can be observed, demonstrating that the directional derivative depends strongly on the chosen direction.The directional derivative is computed using the gradient...
01:26Linear Approximation in Frequency Domain
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear.
01:23Linear Approximations
For a differentiable function of two variables, linear approximation estimates values near a known point by replacing the curved surface with its tangent plane. Consider the function\begin{equation*}f(x,y)=x^2+3y^2\end{equation*}near the point (2, 1). The exact value at this point is f(2, 1) = 22 + 3(1)2 = 4 + 3 = 7.The linear approximation of f(x, y)) near (a, b) is\begin{equation*}L(x,y)=f(a,b)+f_x(a,b)(x-a)+f_y(a,b)(y-b)\end{equation*}First, compute the partial derivatives: fx(x, y) = 2x and...
01:18Reducing Line Loss
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss in...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss in...
01:26Linearization and Approximation
Linearization is a mathematical technique used to approximate complex, nonlinear functions with simpler linear models in the vicinity of a chosen reference point. The method is based on the idea that, although a function may be difficult to evaluate exactly, its behavior near a specific input value can often be closely approximated by the tangent line at that point. This approach is particularly useful when small deviations from a known value are involved.Consider the square root function, for...
02:18Fischer Projections
Learning to draw Fischer projections of molecules and understanding their relevance plays a crucial role in the visual depiction of organic molecules. A Fischer projection is a two-dimensional projection on a planar surface to simplify the three-dimensional wedge–dash representation of molecules. This is especially helpful in the case of molecules with multiple chiral centers that can be difficult to draw. Here, all the bonds of interest are represented as horizontal or vertical lines. While...