GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Information

Area of Science:

• Genomics
• Bioinformatics
• Artificial Intelligence

Background:

• Large language models (LLMs) struggle with factual accuracy (hallucinations) in specialized domains.
• Integrating LLMs with domain-specific tools like databases improves knowledge access and precision.
• Genomics research requires accurate retrieval and interpretation of vast biological data.

Approach:

• We introduce GeneGPT, a method to train LLMs to utilize National Center for Biotechnology Information (NCBI) Web APIs.
• GeneGPT employs in-context learning and an augmented decoding algorithm to enable Codex to interact with NCBI APIs.
• The system is designed to detect and execute API calls for answering genomics-related queries.

Key Points:

• GeneGPT achieves state-of-the-art performance on the GeneTuring benchmark, outperforming existing LLMs and retrieval-augmented models.
• API demonstrations are more effective than documentation for in-context learning across tasks.
• GeneGPT demonstrates generalization to multi-hop questions and longer API call sequences in the novel GeneHop dataset.

Conclusions:

• GeneGPT offers a significant advancement in applying LLMs to genomics by enabling precise data retrieval via NCBI APIs.
• The findings highlight the effectiveness of API integration and in-context learning for specialized LLM applications.
• Further analysis provides insights into error types, guiding future LLM development in bioinformatics.