Chemical Language Model Linker: blending text and molecules with modular adapters

Area of Science:

• Computational chemistry
• Artificial intelligence
• Drug discovery

Background:

• Large language models (LLMs) and multi-modal models offer potential for generating novel molecules from text descriptions.
• Current multi-modal models often require training from scratch, consuming significant computational resources and limiting scalability.
• Existing methods lack efficient integration of pretrained text and molecular models.

Purpose of the Study:

• To propose a novel, lightweight adapter-based strategy (ChemLML) for conditional molecular generation from text descriptions.
• To enable the use of diverse pretrained text models for molecule generation without extensive retraining.
• To investigate the impact of molecular representations (SMILES vs. SELFIES) on generation performance.

Main Methods:

• Developed ChemLML, a lightweight adapter-based strategy that blends pretrained text and molecular models.
• Operated within specialized embedding spaces of the molecular domain to maintain domain specificity.
• Trained minimal adapter parameters to tailor diverse pretrained text models for molecule generation.
• Evaluated the influence of SMILES and SELFIES molecular representations on conditional generation.

Main Results:

• ChemLML effectively generates molecules from text descriptions by leveraging pretrained models.
• Adapter-based training requires significantly fewer parameters compared to training from scratch.
• The choice of molecular representation (SMILES preferred over SELFIES) impacts generation performance.
• Identified issues with the PubChem dataset for evaluation and provided a filtered test set.

Conclusions:

• ChemLML offers an efficient and scalable approach to conditional molecular generation from text.
• The adapter-based strategy significantly reduces computational overhead and enhances model flexibility.
• Molecular representation choice is critical for optimizing generative model performance.
• Demonstrated practical applications by generating candidate protein inhibitors and membrane-permeable molecules.