Related Concept Videos
01:22Upsampling
311
Managing signal sampling rates is essential in digital signal processing to maintain signal integrity. A decimated signal, characterized by a reduced frequency range due to its lower sampling rate, can be upsampled by inserting zeros between each sample. This upsampling process expands the original spectrum and introduces repeated spectral replicas at intervals dictated by the new Nyquist frequency. To refine this zero-inserted sequence, it is passed through a lowpass filter with a cutoff...
311
01:26Linear Approximation in Frequency Domain
131
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
131
01:18Parseval's Theorem
653
Parseval's theorem is a fundamental concept in signal processing and harmonic analysis. It asserts that for a periodic function, the average power of the signal over one period equals the sum of the squared magnitudes of all its complex Fourier coefficients. This theorem, named after Marc-Antoine Parseval, provides a powerful tool for analyzing the energy distribution in signals.
Interestingly, Parseval's theorem also holds for the trigonometric form of the Fourier series, which...
Interestingly, Parseval's theorem also holds for the trigonometric form of the Fourier series, which...
653
01:21Linear Approximation in Time Domain
125
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
125
01:15Sampling Theorem
766
In signal processing, the analysis of continuous-time signals, denoted as x(t), often involves sampling techniques to convert these signals into discrete-time signals. This process is essential for digital representation and manipulation. A critical component in sampling is the train of impulses, characterized by the sampling interval and the sampling frequency. The relationship between these parameters and the original signal's properties dictates the success of the sampling process.
766
01:11Sampling Continuous Time Signal
350
In signal processing, a continuous-time signal can be sampled using an impulse-train sampling technique, followed by the zero-order hold method. Impulse-train sampling involves the use of a periodic impulse train, which consists of a series of delta functions spaced at regular intervals determined by the sampling period. When a continuous-time signal is multiplied by this impulse train, it generates impulses with amplitudes corresponding to the signal's values at the sampling points.
In the...
In the...
350
You might also read
Related Articles
Articles linked to this work by shared authors, journal, and citation graph.
Sort by
Same author
Prior-Guided Attribution of Deep Neural Networks for Obstetrics and Gynecology.
IEEE journal of biomedical and health informatics·2023
Same author
RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Merging.
IEEE transactions on pattern analysis and machine intelligence·2022
Same author
Contributions of Artificial Intelligence Reported in Obstetrics and Gynecology Journals: Systematic Review.
Journal of medical Internet research·2022
Related Experiment Video
Updated: Sep 12, 2025
07:28
Psychophysically-anchored, Robust Thresholding in Studying Pain-related Lateralization of Oscillatory Prestimulus Activity
Published on: January 21, 2017
7.1K
NUPES: Non-Uniform Post-Training Quantization via Power Exponent Search.
Summary
This study introduces NUPES, a novel non-uniform quantization method for deep neural networks (DNNs) and large language models (LLMs). NUPES optimizes quantization parameters during training, achieving state-of-the-art compression rates for efficient model deployment.
Area of Science:
- Artificial Intelligence
- Machine Learning
- Computer Engineering
Background:
- Deep neural network (DNN) deployment is limited by high computational costs, especially for large language models (LLMs).
- Quantization, converting floating-point to fixed-point representations, reduces memory and latency but uniform methods struggle with non-bell-shaped DNN weight/activation distributions and LLM outliers.
- Existing post-training quantization techniques are insufficient for optimizing quantization parameters like exponents and weights effectively.
Purpose of the Study:
- To propose NUPES, an advanced non-uniform quantization technique to overcome limitations of uniform quantization in DNNs and LLMs.
- To develop a novel training paradigm for optimizing quantization operators and weights within the entire quantized space.
- To enable integer-only, low-bit inference while maintaining model performance and achieving high compression rates.
Main Methods:
- NUPES leverages power function-derived automorphisms to preserve scalar multiplications during quantization.
- A new training paradigm learns quantized weights across the entire quantized space and optimizes the quantization operator's exponent parameter.
- Numerical instabilities are alleviated, enabling end-to-end training of the quantization process.
Main Results:
- NUPES achieves state-of-the-art compression rates in both data-free and data-driven quantization configurations.
- The method effectively addresses the limitations of uniform quantization for distributions with outliers, particularly in transformers and LLMs.
- Empirical benchmarks demonstrate NUPES's superior performance compared to previous post-training quantization techniques.
Conclusions:
- NUPES offers a significant advancement in non-uniform quantization for efficient DNN and LLM deployment.
- The proposed training paradigm successfully optimizes quantization parameters, leading to superior compression and performance.
- NUPES provides a viable solution for deploying large models on resource-constrained hardware.