Curiosity-Driven Exploration with Information Bottleneck Representations and Matrix-Based Mutual Information

Curiosity empowers humans to ask questions about the world and explore it without relying on extrinsic, encouraging rewards such as money. To investigate how this mechanism drives exploration, we implement a curiosity-based approach and test it in a reinforcement learning environment. We define curiosity using a hybrid intrinsic signal based on prediction error and the rarity of state-action pairs. To address the curse of dimensionality in raw pixel inputs, we adopt the Information Bottleneck (IB) principle to learn low-dimensional representations that are both compact and predictive. We introduce two formulations for computing mutual information-one based on entropy decomposition and the other on matrix-based Rényi entropy-and compare their effectiveness. Experiments on Acrobot show substantially improved exploration efficiency over Intrinsic Curiosity Module (ICM), Random Network Distillation (RND), and a k-NN novelty baseline, while results on MountainCar indicate that the proposed method is not uniformly superior in low-dimensional environments. These findings suggest that IB-shaped representations and matrix-based information objectives are most beneficial when observations are high-dimensional or dynamics are structurally complex.