Failure Event Mining With Fine-Tuned Large Language Model: Case Study of Analyzing United States Nuclear Power Plant...

Area of Science:

• Engineering
• Computer Science
• Data Science

Background:

• Failure event narratives offer crucial insights into failure initiation and propagation.
• Analyzing event causality aids in understanding failure physics and utilizing non-failure data.
• Automated causality extraction faces challenges due to complex language and limited annotated datasets.

Purpose of the Study:

• To develop a novel large language model (LLM)-based approach for automated causality extraction from text.
• To leverage LLM capabilities for understanding intricate language patterns and long-range contexts.
• To accurately extract cause-and-effect pairs from failure event reports.

Main Methods:

• A twofold framework involving causality detection and causality extraction was proposed.
• A deep learning model was trained for identifying texts containing causality.
• A T5-based LLM (T5-CE) was developed for extracting cause-and-effect pairs.
• A large, annotated dataset of U.S. nuclear power plant failure event reports was utilized for training and evaluation.

Main Results:

• The proposed LLM-based approach demonstrated effective detection of implicit and embedded causalities.
• The models achieved strong performance in identifying cause-and-effect pairs within text.
• Evaluation metrics including precision, recall, and F1 score were used to assess model performance.

Conclusions:

• The novel LLM approach significantly enhances automated causality extraction from failure event narratives.
• This method can improve failure analysis by accurately identifying causal relationships.
• The approach offers a valuable tool for high-reliability industries with limited failure data.