Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods.

Area of Science:

• Artificial Intelligence
• Machine Learning

Background:

• Large language models (LLMs) possess extensive pretrained knowledge and general capabilities.
• Reinforcement learning (RL) benefits from LLMs in areas like multitask learning, sample efficiency, and task planning.

Purpose of the Study:

• To provide a comprehensive review of LLM-enhanced RL literature.
• To propose a structured taxonomy for categorizing LLM functionalities in RL.
• To clarify research scope and future directions in LLM-enhanced RL.

Main Methods:

• Systematic review of existing literature on LLM-enhanced RL.
• Development of a taxonomy categorizing LLMs into four roles: information processor, reward designer, decision-maker, and generator.
• Analysis of methodologies, mitigated RL challenges, and future insights for each role.

Main Results:

• LLMs can be categorized into four distinct roles within the RL framework.
• LLM-enhanced RL addresses challenges in multitask learning, sample efficiency, and high-level planning.
• Comparative analysis of LLM roles, applications, opportunities, and challenges is presented.

Conclusions:

• The proposed taxonomy offers a framework for leveraging LLMs in RL.
• LLM-enhanced RL has the potential to accelerate applications in robotics, autonomous driving, and energy systems.
• Future research should focus on exploring the diverse applications and addressing the challenges of LLM integration in RL.