Leveraging large language models for data analysis automation.

Area of Science:

• Bioinformatics
• Computational Biology
• Genomics

Background:

• Data analysis in biology is crucial but limited by expert shortages.
• Large Language Models (LLMs) show promise for automating code generation.
• The accuracy of LLMs for specialized omics data analysis remains an open question.

Purpose of the Study:

• To develop and evaluate an R package, mergen, that uses LLMs for generating and executing data analysis pipelines.
• To enable researchers to perform complex data analysis through natural language descriptions.
• To investigate the effectiveness of prompt engineering and self-correction mechanisms for improving LLM-generated code accuracy.

Main Methods:

• Developed the mergen R package integrating LLMs for data analysis code generation and execution.
• Employed specialized prompt engineering and error feedback mechanisms to enhance code quality.
• Evaluated performance on various genomics data analysis tasks with different complexity levels.
• Utilized a self-correction strategy to iteratively improve code generation.

Main Results:

• LLMs can generate code for some data analysis tasks, but challenges persist for complex analyses.
• The self-correction mechanism significantly improved executable code generation rates across task complexities (22.5% to 52.5%).
• Statistical analysis confirmed significant differences in performance across prompting strategies.

Conclusions:

• LLMs show potential for automating bioinformatics data analysis, but require careful implementation.
• The mergen package and its self-correction feature offer a practical approach to improve LLM-driven code generation.
• Further research is needed to fully address LLM limitations in complex, domain-specific data analysis.