Extracting accurate materials data from research papers with conversational language models and prompt engineering.

Area of Science:

• Natural Language Processing (NLP)
• Artificial Intelligence (AI)
• Materials Science Data Management

Background:

• Manual data extraction from research papers is time-consuming and labor-intensive.
• Existing automated methods, including early language models, require significant expertise and coding.
• Large Language Models (LLMs) offer potential for improved automated data extraction but face challenges with accuracy.

Purpose of the Study:

• To introduce ChatExtract, a novel method for fully automated, accurate data extraction from research papers.
• To minimize the initial effort and technical background required for data extraction.
• To leverage conversational LLMs for efficient and reliable data retrieval.

Main Methods:

• Engineered prompts applied to conversational LLMs to identify and extract data.
• Utilized follow-up questions to verify data accuracy and mitigate LLM factual errors.
• Tested ChatExtract on materials science data, including critical cooling rates and yield strengths.

Main Results:

• ChatExtract achieved high data extraction accuracy, with precision and recall close to 90% using advanced LLMs like GPT-4.
• The method demonstrated effectiveness in overcoming LLM factual inaccuracies through conversational verification.
• Developed databases for metallic glasses' critical cooling rates and high entropy alloys' yield strengths.

Conclusions:

• ChatExtract offers a simple, transferable, and accurate approach to automated data extraction.
• Conversational LLMs, combined with strategic prompting, significantly enhance data extraction reliability.
• This method is poised to become a powerful tool for data extraction in scientific research.