Gene Set Summarization Using Large Language Models.

Area of Science:

• Bioinformatics
• Computational Biology
• Genomics

Background:

• Molecular biologists analyze gene lists from high-throughput experiments using statistical enrichment analysis.
• Gene Ontology (GO) is a key knowledge base for annotating gene functions.
• Interpreting gene lists can be approached as a textual summarization task.

Purpose of the Study:

• To evaluate Large Language Models (LLMs) for gene set function summarization as a complement to standard enrichment analysis.
• To assess LLM performance using curated annotations, narrative summaries, or direct model retrieval.
• To compare LLM-based summarization with traditional statistical methods for gene list interpretation.

Main Methods:

• TALISMAN (Terminological ArtificiaL Intelligence SuMmarization of Annotation and Narratives) was developed using generative AI.
• LLMs were tested with different data sources: ontological annotations, narrative summaries, and direct retrieval.
• Performance was evaluated based on the plausibility, biological validity, precision, and recall of generated GO term lists.

Main Results:

• LLM-based methods generated plausible and biologically valid GO term lists.
• LLMs could not provide reliable statistical scores (p-values) and often returned non-significant terms.
• LLMs rarely recapitulated the most precise terms identified by standard enrichment analysis.
• Newer LLM models demonstrated statistically significant improvements over older models.
• Prompt variations led to non-deterministic and radically different term lists.

Conclusions:

• LLM-based methods are currently unsuitable as a replacement for standard gene enrichment analysis.
• LLMs may offer summarization benefits for integrating implicit knowledge and processing large, complex gene sets.
• Future advancements in LLM technology may enhance their utility in biological data interpretation.