Integrating visual large language model and reasoning chain for driver behavior analysis and risk assessment

Area of Science:

• Computer Science
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Driver behavior significantly impacts road safety.
• Sophisticated methods for classifying driver distractions are essential.
• Existing models may lack interpretability and reasoning capabilities.

Purpose of the Study:

• To develop a novel Distracted Driving Classification (DDC) approach using a visual Large Language Model (LLM).
• To enhance driver distraction detection and classification accuracy.
• To improve the interpretability and reasoning behind distraction assessments.

Main Methods:

• Utilized a visual Large Language Model (LLM) named the Distracted Driving Language Model (DDLM).
• Incorporated whole-body human pose estimation to analyze key postural features (head, hands).
• Integrated a reasoning chain framework to provide explanations for classifications.

Main Results:

• The DDLM demonstrated enhanced performance in classifying driver behaviors and associated risk levels.
• Achieved superior results in both zero-shot and few-shot learning scenarios on the 100-Driver dataset.
• Provided detailed, context-aware evaluations of driver distractions.

Conclusions:

• The DDLM is an advanced tool for accurately detecting and analyzing driving distractions.
• The approach shows significant potential for improving overall driving safety.
• The integration of pose estimation and reasoning enhances LLM capabilities for this task.