相关实验视频
Updated: Jan 8, 2026

06:42
Generation and Coherent Control of Pulsed Quantum Frequency Combs
Published on: June 8, 2018
9.6K
实现基于KV缓存压缩的矩阵产品运算符的高效低位量化.
Jia-Qi Wang1, Xiao-Qi Han1, Peng-Jie Guo1
1School of Physics, Renmin University of China, Beijing, China.
概括
我们开发了MPOQ,一种新的无数据量化方法,用于压缩大型语言模型 (LLM) 键值 (KV) 缓存. 这种技术通过智能定量张量来显著降低内存使用量,提高了LLM效率而不牺牲准确性.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算机科学 计算机科学
背景情况:
- 大型语言模型 (LLM) 的部署在计算上昂贵.
- 关键值 (KV) 缓存对于LLM推断速度至关重要,但需要大量的内存.
- 现有的KV缓存压缩方法通常会损害准确性或需要校准数据.
研究的目的:
- 引入MPOQ,一种新的无数据量子化技术,用于压缩LLM KV缓存.
- 在不降低性能的情况下减少LLM的内存足迹.
- 为现实世界LLM应用提供实用解决方案.
主要方法:
- 开发了MPOQ,一种使用矩阵产品运算符 (MPO) 的量子化技术.
- MPO将矩阵分解为局部张量,使目标定量化成为可能.
- 采用混合量子化策略:大张量器的低位,含异常值的较小张量器的高精度.
主要成果:
- 在KV缓存内存足迹中实现了大约75%的减少.
- 保持了与未压缩模型相比较的发电质量.
- 在包括OPT,LLaMA和Mistral在内的各种LLM中表现出有效性.
结论:
- MPOQ提供了一种有效的无数据方法来压缩LLM KV缓存.
- 该方法显著提高了LLM的效率,并降低了内存成本.
- MPOQ为实际的LLM部署提出了一个可行的策略.
相关概念视频
Vector Representation of Complex Numbers
449
Complex numbers, represented in Cartesian coordinates, can also be visualized as vectors. These vectors can be expressed in polar form, emphasizing their magnitude and angle. When a complex number is input into a function, the output is another complex number, highlighting the function's zero point from which the vector representation can originate.
Consider a function defined as the product of the complex factors in the numerator divided by the product of the complex factors in the...
Consider a function defined as the product of the complex factors in the numerator divided by the product of the complex factors in the...
449
Extraction: Partition and Distribution Coefficients
4.5K
The distribution law or Nernst's distribution law is the law that governs the distribution of a solute between two immiscible solvents. This law, also known as the partition law, states that if a solute is added to the mixture of two immiscible solvents at a constant temperature, the solute is distributed between the two solvents in such a way that the ratio of solute concentrations in the solvents remains constant at equilibrium.
For extracting a solute from an aqueous phase into an...
For extracting a solute from an aqueous phase into an...
4.5K
Vector Operations
1.9K
Vectors are physical quantities that have both magnitude and direction. The vector operations include addition, subtraction, and scalar multiplication.
A vector multiplied by a scalar value is called scalar multiplication. The result obtained is a new vector with a different magnitude. If the scalar is positive, the direction of the vector remains the same, but if it is negative, the direction of the vector is reversed. For example, the product of the mass and velocity yields the momentum.
A vector multiplied by a scalar value is called scalar multiplication. The result obtained is a new vector with a different magnitude. If the scalar is positive, the direction of the vector remains the same, but if it is negative, the direction of the vector is reversed. For example, the product of the mass and velocity yields the momentum.
1.9K
Vector Algebra: Method of Components
18.7K
It is cumbersome to find the magnitudes of vectors using the parallelogram rule or using the graphical method to perform mathematical operations like addition, subtraction, and multiplication. There are two ways to circumvent this algebraic complexity. One way is to draw the vectors to scale, as in navigation, and read approximate vector lengths and angles (directions) from the graphs. The other way is to use the method of components.
In many applications, the magnitudes and directions of...
In many applications, the magnitudes and directions of...
18.7K
Linear Approximation in Frequency Domain
329
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
329
Phasor Arithmetics
707
Phasors and their corresponding sinusoids are interrelated, offering unique insights into the behavior of alternating current (AC) circuits. One way to understand this relationship is through the operations of differentiation and integration in both the time and phasor domains.
When the derivative of a sinusoid is taken in the time domain, it transforms into its corresponding phasor multiplied by j-omega (jω) in the phasor domain, where j is the imaginary unit, and ω is the angular...
When the derivative of a sinusoid is taken in the time domain, it transforms into its corresponding phasor multiplied by j-omega (jω) in the phasor domain, where j is the imaginary unit, and ω is the angular...
707

