Related Experiment Videos
A model for learning to segment temporal sequences, utilizing a mixture of RNN experts together with adaptive
1Brain Science Institute, RIKEN, 2-1 Hirosawa, Wako-shi, Saitama, 351-0198, Japan. jnamika@bdc.brain.riken.go.jp
Summary
This study introduces a new recurrent neural network (RNN) mixture of experts model that dynamically switches experts for sequence generation. This novel method improves learning performance and successfully models complex time series data.
Area of Science:
- Artificial Intelligence
- Machine Learning
- Robotics
Background:
- Recurrent Neural Networks (RNNs) are effective for sequence modeling.
- Mixture of Experts (MoE) models offer enhanced capacity by combining specialized networks.
- Dynamic switching between experts is crucial for complex sequence generation tasks.
Purpose of the Study:
- To propose a novel learning method for a mixture of RNN experts model.
- To enable dynamic expert switching for desired sequence generation.
- To improve upon conventional methods in learning complex time series.
Main Methods:
- Utilizing maximum likelihood estimation with a gradient descent algorithm.
- Modifying the likelihood function with an expert-specific variance alteration mechanism.
- Incorporating a gating network for realistic application.
Main Results:
- Successfully learned Markov chain switching for 9 Lissajous curves, outperforming conventional methods.
- Demonstrated superior generalization capability compared to conventional approaches.
- Applied successfully to learning sensory-motor flows for a humanoid robot.
Conclusions:
- The proposed RNN mixture of experts model with dynamic switching offers superior performance.
- The method effectively handles complex time series prediction and generation.
- This approach has practical implications for robotics and advanced sequence modeling.
