A scientific-article key-insight extraction system based on multi-actor of fine-tuned open-source large language...

Area of Science:

• Artificial Intelligence
• Computational Linguistics
• Bibliometrics

Background:

• The rapid increase in scientific publications necessitates automated methods for information organization and extraction.
• Manual literature reviews are time-consuming and struggle to keep pace with research output.

Purpose of the Study:

• To investigate the efficacy of Large Language Models (LLMs) for extracting key insights from scientific articles.
• To develop and evaluate an LLM-based system (ArticleLLM) for article-level key-insight extraction.
• To enhance LLM performance through fine-tuning and explore a multi-actor LLM approach.

Main Methods:

• Evaluation of leading LLMs (GPT-4.0, Mixtral 8x7B, Yi, InternLM2) against manual extraction benchmarks.
• Development of the ArticleLLM system utilizing fine-tuned LLMs.
• Implementation of a multi-actor LLM strategy to combine the strengths of individual models.

Main Results:

• LLMs demonstrate significant potential for accurate key-insight extraction from scientific literature.
• Fine-tuning improves the performance of individual LLMs for this task.
• A multi-actor approach, merging multiple fine-tuned LLMs, yields superior extraction performance compared to single models.

Conclusions:

• LLMs are feasible and effective tools for automating key-insight extraction in scientific articles.
• Cooperation among multiple fine-tuned LLMs enhances the efficiency of academic literature surveys and knowledge discovery.
• The ArticleLLM system and multi-actor approach offer a promising direction for managing and synthesizing scientific information.