DrugPlayGround: Benchmarking Large Language Models and Embeddings for Drug Discovery

Area of Science:

• Computational chemistry and pharmacology
• Artificial intelligence in drug discovery

Background:

• Large language models (LLMs) are increasingly utilized in drug discovery research.
• There is a need for objective performance assessments of LLMs compared to traditional methods.

Purpose of the Study:

• To develop a framework for evaluating LLM performance in drug discovery.
• To benchmark LLMs for generating text-based descriptions of drug characteristics and interactions.

Main Methods:

• Developed DrugPlayGround, a framework for LLM evaluation.
• Benchmarked LLMs on generating descriptions of physiochemical properties, drug synergism, and drug-protein interactions.
• Incorporated domain expert validation for LLM reasoning.

Main Results:

• DrugPlayGround enables objective assessment of LLM performance in drug discovery tasks.
• The framework tests LLMs' chemical and biological reasoning capabilities.
• Identified advantages and limitations of LLMs in drug research pipelines.

Conclusions:

• DrugPlayGround facilitates the advancement of LLMs in drug discovery.
• Objective evaluation is crucial for integrating LLMs across all stages of drug research.
• Enhanced LLM reasoning capabilities can accelerate hypothesis generation and candidate prioritization.