A case study using sewage metagenomic data for assessment of text-to-SQL capabilities in large language models

Area of Science:

• Bioinformatics
• Computational Biology
• Data Science

Background:

• Relational databases are powerful for complex data but require SQL expertise, posing a barrier for life science users.
• Metagenomic analysis generates large, intricate datasets that are challenging to query using traditional methods.

Purpose of the Study:

• To evaluate the efficacy of a large language model (LLM) in translating plain English queries into SQL scripts (Text-to-SQL).
• To assess the potential of LLMs to simplify database interaction and overcome usage barriers in life sciences.

Main Methods:

• A complex database of metagenomic analyses from 239 sewage samples across five European cities was utilized.
• A specialized LLM, "SewageGPT", was provided with database structure and content details.
• The LLM's accuracy in responding to complex questions and visualizing results was evaluated.

Main Results:

• SewageGPT accurately responded to complex database inquiries when provided with detailed descriptions, accelerating the querying process.
• Knowledge of database content minimized query ambiguities, but ambiguities could lead to incorrect responses.
• Human oversight is essential for ensuring accuracy, particularly with vague or ambiguous queries.

Conclusions:

• Integrating LLMs with direct database connectivity significantly enhances efficiency in query generation, statistical analysis, and result visualization.
• LLMs offer a promising solution to democratize access to complex biological databases for researchers without extensive SQL knowledge.
• Careful prompt engineering and human validation are critical for reliable LLM-powered data analysis.