FRM-PTQ:特征关系匹配对大型语言模型进行增强的低位培训后量化.
Chao Zeng1, Jiaqi Zhao1, Miao Zhang1
1Harbin Institute of Technology (Shenzhen), Shen Zhen, 518057, Guang Dong, China.
概括
在大型语言模型中,FRM-PTQ通过匹配特征关系来显著改善低位后训练量化 (PTQ),保持准确性,同时提高吞吐量和减少内存使用.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算机科学 计算机科学
背景情况:
- 训练后量化 (PTQ) 对于减少大型语言模型 (LLM) 推断中的内存和计算至关重要.
- 现有的PTQ方法与超低位量化作斗争,导致性能降低,特别是在LLaMA-3和LLaMA-3.1.1等先进模型中.
研究的目的:
- 开发一种新的PTQ框架,FRM-PTQ,以减轻超低位量子化中的性能损失.
- 通过先进的量子化技术来提高LLM的效率和准确性.
主要方法:
- 引入特征关系匹配 (FRM) 以通过区块内部自蒸来建模令牌级关系和结构级分布.
- 在高维空间中使用特征关系匹配来弥合量子化和全精度块之间的差距,超越传统的MSE损失.
- 开发了一种基于解码器块灵敏度的定制内核的多颗粒度每组量子化技术.
主要成果:
- 在W4A4低位量子化中,FRM-PTQ实现了卓越的性能,保持了接近完全精度的准确性.
- 证明了2倍的吞吐量改善和3.17倍的内存减少.
- 在最新的模型 (LLaMA-3,LLaMA-3.1,Qwen2.5) 和极低位场景 (W3A3) 中展示了显著的优势.
结论:
- FRM-PTQ有效地解决了LLMs的超低位量子化中的性能降解问题.
- 拟议的特征关系匹配和多颗粒度定量化为高效的LLM部署提供了强大的解决方案.
- FRM-PTQ提供了一种实用和有效的方法,可以在减少资源需求的情况下部署最先进的LLM.
更多相关视频
相关概念视频
Relationship Formation
45.3K
What do you think is the single most influential factor in determining with whom you become friends and whom you form romantic relationships? You might be surprised to learn that the answer is simple: the people with whom you have the most contact. This most important factor is proximity. You are more likely to be friends with people you have regular contact with. For example, there are decades of research that shows that you are more likely to become friends with people who live in your dorm,...
45.3K
Language
900
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
900
Sign Test for Matched Pairs
394
The sign test for matched pairs offers a robust method for comparing two paired samples, often for the effects of an intervention in one of them. This method is very useful in situations where the underlying distribution of the data is unknown. The test compares two related samples—often pre- and post-treatment measurements on the same subjects—to determine if there are significant differences in their median values.
To conduct the sign test, we first calculate the differences in...
To conduct the sign test, we first calculate the differences in...
394
Ending Relationships
178
The dissolution of intimate relationships presents complex emotional and psychological challenges, particularly when emotional bonds are strong, the relationship is long-standing, and perceived alternatives are limited. This distress often intensifies in romantic breakups, where the initiator may experience greater turmoil than the rejected partner. Contributing factors include residual attachment, guilt over causing pain, and uncertainty about how to manage the situation. The stress is further...
178
Components of Language
809
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
809
Language Development
880
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
880


