Integrating chemistry knowledge in large language models via prompt engineering.

Area of Science:

• Artificial Intelligence
• Computational Chemistry
• Materials Science

Background:

• Large language models (LLMs) show promise for scientific applications but require specialized knowledge for optimal performance.
• Traditional prompt engineering methods may not fully leverage LLMs' potential in complex scientific domains.
• Integrating domain-specific information into prompts is crucial for enhancing LLM accuracy and relevance in scientific contexts.

Purpose of the Study:

• To investigate the impact of domain-specific knowledge integration in prompt engineering for large language models (LLMs).
• To develop and evaluate a domain-knowledge embedded prompt engineering method for scientific applications.
• To compare the proposed method against traditional prompt engineering strategies using key performance metrics.

Main Methods:

• Development of a novel domain-knowledge embedded prompt engineering technique.
• Application of the method to complex scientific case studies, including the MacMillan catalyst, paclitaxel, and lithium cobalt oxide.
• Quantitative evaluation using metrics such as capability, accuracy, F1 score, and hallucination reduction.

Main Results:

• The domain-knowledge embedded prompt engineering method demonstrated superior performance compared to traditional approaches.
• Significant improvements were observed in accuracy, F1 score, and a notable reduction in hallucinations.
• Case studies confirmed the method's effectiveness in handling complex scientific materials and generating reliable outputs.

Conclusions:

• Domain-specific prompts effectively guide LLMs to produce more accurate and relevant scientific information.
• The proposed method enhances LLMs' utility as powerful tools for scientific discovery and innovation.
• Further research into domain-specific prompt engineering is warranted to unlock the full potential of LLMs in science.