Modular learning models in forecasting natural phenomena

Modular model is a particular type of committee machine and is comprised of a set of specialized (local) models each of which is responsible for a particular region of the input space, and may be trained on a subset of training set. Many algorithms for allocating such regions to local models typically do this in automatic fashion. In forecasting natural processes, however, domain experts want to bring in more knowledge into such allocation, and to have certain control over the choice of models. This paper presents a number of approaches to building modular models based on various types of splits of training set and combining the models' outputs (hard splits, statistically and deterministically driven soft combinations of models, 'fuzzy committees', etc.). An issue of including a domain expert into the modeling process is also discussed, and new algorithms in the class of model trees (piece-wise linear modular regression models) are presented. Comparison of the algorithms based on modular local modeling to the more traditional 'global' learning models on a number of benchmark tests and river flow forecasting problems shows their higher accuracy and transparency of the resulting models.