Fusing Domain Knowledge with a Fine-Tuned Large Language Model for Enhanced Molecular Property Prediction

Area of Science:

• Computational Chemistry
• Drug Discovery
• Artificial Intelligence

Background:

• Large language models (LLMs) show promise in scientific applications but struggle with molecular property prediction.
• Existing chemistry-specific LLMs have not achieved satisfactory performance in this crucial drug discovery task.

Purpose of the Study:

• To enhance molecular property prediction accuracy by integrating profound domain knowledge into LLMs.
• To develop a novel dual-modality learning approach for improved drug discovery predictions.

Main Methods:

• Fine-tuned DeepSeek-R1-Distill-Qwen-1.5B using ZINC and ChEMBL datasets to obtain SMILES embeddings.
• Integrated LLM-derived SMILES embeddings with molecular graph representations.
• Trained a hybrid neural network on combined dual-modality inputs for property prediction.

Main Results:

• The Knowledge-Fused Large Language Model for dual-Modality (KFLM2) achieved higher prediction performance on nine out of ten regression and classification datasets.
• Visualizations confirmed that combining LLM embeddings with molecular graphs provides complementary information, boosting prediction accuracy.
• Model performance was not solely dependent on size but on effective knowledge integration from pretraining and fine-tuning.

Conclusions:

• Integrating domain knowledge into LLMs is a rational and effective strategy for precise molecular property prediction.
• The proposed KFLM2 method offers a significant advancement for revolutionizing drug development and discovery processes.
• Dual-modality learning, combining LLM embeddings and molecular graphs, enhances predictive capabilities beyond single-modality approaches.