A Hybrid Transformer Architecture with a Quantized Self-Attention Mechanism Applied to Molecular Generation

Area of Science:

• Quantum Computing
• Artificial Intelligence
• Computational Chemistry

Background:

• Self-attention mechanisms are crucial for transformer architectures, powering large language models (LLMs).
• Classical self-attention involves computationally intensive query, key, and value matrix operations.
• Reducing computational overhead in attention mechanisms is a key challenge for scaling AI models.

Purpose of the Study:

• To propose and evaluate a hybrid quantum-classical self-attention mechanism.
• To integrate this mechanism into a transformer decoder architecture for LLMs.
• To demonstrate its application in conditional generation tasks within computational chemistry.

Main Methods:

• Developed a hybrid quantum-classical self-attention mechanism.
• Integrated the mechanism into a transformer decoder.
• Trained the model on the QM9 dataset using SMILES strings and physicochemical properties for conditional generation.
• Performed simulations using NVIDIA's CUDA-Q platform.

Main Results:

• Theoretical analysis shows a reduction in time complexity for the query-key dot product from O(n^2d) to O(n log n).
• The hybrid model demonstrated utility in conditional generation tasks on the QM9 dataset.
• Simulations on CUDA-Q highlighted efficient GPU scalability.

Conclusions:

• The proposed hybrid quantum-classical self-attention mechanism offers a promising approach for quantum-enhanced natural language processing.
• This method has the potential to significantly reduce computational costs in LLMs.
• The approach shows utility in scientific applications like computational chemistry.