Information-Theoretic Intrinsic Motivation for Reinforcement Learning in Combinatorial Routing

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Theory

Background:

• Intrinsic motivation is key for reinforcement learning (RL) exploration when rewards are scarce.
• Defining novelty in high-dimensional state spaces is challenging for traditional RL methods.

Purpose of the Study:

• To propose an information-theoretic framework for intrinsically motivated RL using the Information Bottleneck principle.
• To develop a method for learning compact latent state representations that balance observation compression and predictive information.

Main Methods:

• Utilized the Information Bottleneck principle to create latent state representations.
• Defined intrinsic rewards based on mutual information within the latent space.
• Employed neural mutual information estimators for scalable estimation in high-dimensional settings.

Main Results:

• The proposed method demonstrated improved exploration efficiency.
• Enhanced training stability and solution quality were observed compared to standard RL baselines.
• Effective evaluation on combinatorial routing problems like TSP and SDVRP.

Conclusions:

• Information bottleneck-driven intrinsic motivation offers a principled approach for RL exploration.
• The framework effectively addresses challenges in high-dimensional and combinatorial state spaces.
• This method advances the state-of-the-art in intrinsically motivated reinforcement learning.