Efficient Training Corpus Retrieval for Large Language Model Fine Tuning: A Case Study in Cancer

Area of Science:

• Biomedical Informatics
• Computational Biology
• Cancer Research

Background:

• Current knowledge retrieval in cancer research is limited.
• Automated tools are needed for efficient data collection and analysis.

Purpose of the Study:

• To develop an automated knowledge extraction tool (KnowledgePipeline) for cancer research.
• To build high-quality academic corpora for fine-tuning large language models (LLMs).
• To investigate the tool's effectiveness in interleukin-6 and bladder cancer domains.

Main Methods:

• KnowledgePipeline integrates academic paper content, co-citations, and co-authorship networks.
• Two LLMs (GPTJ-6.7B and Galactica30B) were fine-tuned on domain-specific question-answer pairs.
• Evaluation focused on knowledge extraction quality and fine-tuned model performance in question-answering tasks.

Main Results:

• KnowledgePipeline provides a scalable, automated framework for domain-specific knowledge retrieval.
• High relevance scores were achieved: 68% for IL-6 and 74.5% for bladder cancer.
• A fine-tuned Galactica-30B model demonstrated promising capabilities in question-answering.

Conclusions:

• KnowledgePipeline advances literature discovery and addresses critical biomedical challenges in cancer research.
• The tool facilitates fine-tuned LLM applications for improved cancer research outcomes.
• Automated knowledge extraction is crucial for enhancing LLM capabilities in specialized scientific domains.