Large Language Models in Sensor-Driven Control Systems: Architectures, Challenges, and Opportunities

Area of Science:

• Robotics and Control Systems Engineering
• Artificial Intelligence and Machine Learning
• Cyber-Physical Systems

Background:

• Large language models (LLMs) show potential for integration into diverse sensor-driven control systems.
• Existing research often treats LLMs as generic agents, overlooking their specific role in the sensing-decision-control chain.
• Sensor-driven control systems rely on sensing for monitoring, estimation, supervision, planning, decision-making, and actuation.

Purpose of the Study:

• To review and synthesize emerging research on LLMs in sensor-driven control systems.
• To examine LLM interaction with state representations, supervisory logic, human operators, external tools, and classical control components.
• To develop a functional taxonomy of LLM roles and assess their maturity and deployment risk.

Main Methods:

• Analysis of LLM integration within the sensing-decision-control chain.
• Development of a functional taxonomy based on proximity to actuation, grounding requirements, and deployment risk.
• Identification of key integration patterns and persistent challenges.

Main Results:

• A maturity gradient exists for LLM roles: interpretive, supervisory, diagnostic, and engineering-support roles are most credible.
• Runtime control participation is the least mature and highest-risk integration.
• Reliable implementations are predominantly hybrid, with LLMs acting as semantic and orchestration layers.

Conclusions:

• LLMs are most credible in interpretive, supervisory, diagnostic, and human-facing roles within hybrid architectures.
• Future progress requires neuro-symbolic integration, efficient local deployment, and human-centered autonomy.
• Rigorous evaluation and realistic benchmarks are crucial for addressing challenges like hallucination and weak physical grounding.