一个混合变压器架构与量子化自我注意力机制应用于分子生成
Anthony M Smaldone1, Yu Shee1, Gregory W Kyro1
1Department of Chemistry, Yale University, New Haven, Connecticut 06520, United States.
Journal of chemical theory and computation
|May 7, 2025
概括
研究人员开发了一种混合量子-经典自我注意机制,以增强大型语言模型 (LLM) 的变压器解码器. 这种量子方法显著降低了自然语言处理任务的计算复杂性.
科学领域:
- 量子计算是一种量子计算.
- 人工智能的人工智能
- 计算化学计算化学
背景情况:
- 对于变压器架构来说,自我注意机制至关重要,为大型语言模型 (LLM) 提供动力.
- 经典的自我注意涉及计算密集的查询,键和值矩阵操作.
- 减少注意力机制中的计算开销是扩展人工智能模型的关键挑战.
研究的目的:
- 提出和评估一个混合量子-古典自我注意力机制.
- 将这种机制集成到LLMs的变压器解码器架构中.
- 为了证明其在计算化学中的条件生成任务中的应用.
主要方法:
- 开发了一种混合量子古典自我注意力机制.
- 将该机制集成到变压器解码器中.
- 在QM9数据集上训练模型,使用SMILES字符串和条件生成的物理化学特性.
- 使用NVIDIA的CUDA-Q平台进行模拟.
主要成果:
- 理论分析显示,查询键点积的时间复杂性从O(n^2d) 减少到O(n log n).
- 混合模型在QM9数据集上的条件生成任务中证明了实用性.
- 在CUDA-Q上的模拟突出了高效的GPU可扩展性.
结论:
- 拟议的混合量子-经典自我注意机制为量子增强的自然语言处理提供了一个有希望的方法.
- 这种方法有可能显著降低LLMs的计算成本.
- 这种方法在科学应用中具有实用性,例如计算化学.
相关概念视频
Molecular Orbital Theory I
Overview of Molecular Orbital Theory
Atomic Nuclei: Nuclear Relaxation Processes
In the absence of an external magnetic field, nuclear spin states are degenerate and randomly oriented. When a magnetic field is applied, the spins begin to precess and orient themselves along (lower energy) or against (higher energy) the direction of the field. At equilibrium, a slight excess population of spins exists in the lower energy state. Because the direction of the magnetic field is fixed as the z-axis, the precessing magnetic moments are randomly oriented around the z-axis. This...
π Electron Effects on Chemical Shift: Overview
An applied magnetic field causes loosely bound π-electrons in organic molecules to circulate, producing a local or induced diamagnetic field over a large spatial volume. As the molecules tumble in solution, the field generated by π-electrons in spherical substituents results in a zero net field. However, the net field generated by π-electrons in non-spherical substituents is not zero. The effect of this induced field depends on the orientation of the molecule with respect to B0, resulting in...
Molecular Kinetic Energy
The word "gas" comes from the Flemish word meaning "chaos," first used to describe vapors by the chemist J. B. van Helmont. Consider a container filled with gas, with a continuous and random motion of molecules. During collisions, the velocity component parallel to the wall is unchanged, and the component perpendicular to the wall reverses direction but does not change in magnitude. If the molecule’s velocity changes in the x-direction, then its momentum is changed. During the short time of the...
Van de Graaff Generator
Van de Graaff generators (or Van de Graaffs) are devices used to demonstrate high voltage due to static electricity that can also be used for research. Robert Van de Graaff first built one in 1931 (based on original suggestions by Lord Kelvin) for use in nuclear physics research.
Van de Graaff uses both smooth and pointed surfaces, conductors, and insulators to generate large static charges and, hence, large voltages. A substantial excess charge can be deposited on the sphere because it moves...
Van de Graaff uses both smooth and pointed surfaces, conductors, and insulators to generate large static charges and, hence, large voltages. A substantial excess charge can be deposited on the sphere because it moves...
Carrier Generation and Recombination
Carrier generation is the process by which electron-hole pairs (EHPs) are created within the semiconductor. In direct-bandgap semiconductors, such as gallium arsenide (GaAs), this occurs efficiently when energy absorption prompts valence electrons to leap into the conduction band, leaving behind holes.
This process is given by the generation rate G and is efficient due to the conservation of momentum between the valence band maximum and conduction band minimum.
Indirect generation involves an...
This process is given by the generation rate G and is efficient due to the conservation of momentum between the valence band maximum and conduction band minimum.
Indirect generation involves an...


