Offline reinforcement learning combining generalized advantage estimation and modality decomposition interaction.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Transformers show promise in offline reinforcement learning (RL) for action prediction through trajectory modeling.
• Existing Transformer methods struggle with effective trajectory stitching and capturing deep multimodal interactions.

Purpose of the Study:

• To propose CGM, an offline RL approach enhancing trajectory stitching and multimodal interaction for improved action prediction.
• To address limitations in existing Transformer-based RL methods.

Main Methods:

• CGM combines Generalized Advantage Estimation (GAE) for dataset relabeling and improved trajectory stitching.
• Modality Decomposition Interaction (MDI) employs an encoder (ConvFormer for intra-modal, dual-Transformer for inter-modal) and a decoder.
• The encoder captures intra-modal associations and facilitates deep cross-modal information exchange between states and actions.

Main Results:

• CGM demonstrated superior performance compared to state-of-the-art baseline methods on the D4RL dataset.
• On the MuJoCo dataset, CGM outperformed the optimal comparison method by 2.89%.

Conclusions:

• CGM effectively enhances trajectory stitching and deep multimodal interactions in offline RL.
• The proposed approach shows significant improvements in action prediction accuracy and overall performance.