Learning physical interactions to compose biological large language models

Area of Science:

• Computational biology
• Drug discovery
• Artificial intelligence in medicine

Background:

• Deep learning models, particularly large language models (LLMs), are integral to modern drug design, aiding virtual screening through feature vectors from biochemical sequences.
• Current LLMs lack the ability to fully capture crucial molecular interactions influencing binding affinity and specificity.

Purpose of the Study:

• To address the limitations of existing models by exploring methods to merge diverse molecular representations.
• To propose the development of biochemical foundation models capable of jointly encoding multiple biological data types for enhanced interaction prediction.

Main Methods:

• Overview of existing methods for combining molecular representations from different biological modalities.
• Development and application of a 'composing' strategy for biochemical language models, merging internal layer representations.
• Analysis of recent advancements in interpreting and democratizing LLMs for biological applications.

Main Results:

• The proposed method of 'composing' biochemical language models demonstrates performance comparable to or exceeding standard methods for molecular interaction prediction.
• The composed models achieve this performance with a significantly reduced feature set.
• The study highlights the potential of merging internal representations for improved generalizability in interaction prediction.

Conclusions:

• Merging representations from distinct biological modalities is essential for developing more effective molecular interaction prediction models.
• Future biochemical foundation models should be designed to jointly encode diverse molecular data for comprehensive understanding.
• This approach offers a path towards more accurate and generalizable drug discovery tools, with potential for predicting evolutionary changes in molecular interactions.