LLMs in the Loop: A Survey of Language-Driven Driver Monitoring and Assistance Systems

Area of Science:

• Artificial Intelligence
• Human-Computer Interaction
• Automotive Safety

Background:

• Large language models (LLMs) exhibit human-like reasoning, making them suitable for dynamic driver assistance.
• Current research on LLMs in driver assistance is fragmented, limiting integrated, context-aware support.
• Intelligent driver assistance requires systems that can interpret and adapt to the driver's state.

Purpose of the Study:

• To synthesize advancements in LLM-driven occupant monitoring systems.
• To examine frameworks, benchmarks, and datasets for LLM-based driver assistance.
• To provide a roadmap for developing transparent and trustworthy in-cabin systems.

Main Methods:

• Literature review of recent LLM applications in driver monitoring.
• Critical examination of pioneering frameworks and techniques (e.g., reasoning chains, multimodality).
• Analysis of foundational datasets and human-in-the-loop feedback mechanisms.

Main Results:

• LLMs offer capabilities for interpreting driver states and enabling context-specific actions.
• Techniques like reasoning chains and multimodality enhance personalized and safe driving.
• Existing research highlights trends, limitations, and emerging patterns in LLM-driven systems.

Conclusions:

• LLM-driven occupant monitoring systems represent a new generation of intelligent driver assistance.
• Human-centered evaluation is crucial for trustworthy and transparent in-cabin systems.
• Future research should focus on bridging safety with driver experience through advanced LLM integration.