A large language model framework to uncover underreporting in traffic crashes.

Area of Science:

• Traffic Safety
• Data Science
• Natural Language Processing

Background:

• Traffic crash reports are vital for developing safety countermeasures.
• Underreporting of crash factors due to data collection errors is a significant issue.
• Manual data correction is time-consuming and prone to errors, especially for large datasets.

Purpose of the Study:

• To develop and evaluate a framework for analyzing traffic crash narratives.
• To uncover underreported crash factors using Large Language Models (LLMs).
• To improve the efficiency and accuracy of traffic safety data analysis.

Main Methods:

• The framework integrates prompt engineering, LLM parameter selection, output parsing, and underreporting determination.
• A case study focused on identifying underreported alcohol involvement in traffic crashes.
• Evaluated performance across different LLMs (ChatGPT, Flan-UL2, Llama-2), prompt types, and generation parameters using 500 Massachusetts crash reports.

Main Results:

• The framework achieved high recall (up to 1.0) and precision (up to 0.93) in identifying underreported crash instances.
• Demonstrated efficient and accurate uncovering of underreporting in crash data.
• The approach does not require extensive natural language processing expertise from safety analysts.

Conclusions:

• The developed framework effectively addresses critical gaps in traffic safety analysis.
• Offers a novel method to enhance the quality and comprehensiveness of traffic crash records.
• Paves the way for more effective traffic safety countermeasure development.