A late-stopping method for optimal aggregation of neural networks

Ensembles of artificial neural networks have been used in the last years as classification/regression machines, showing improved generalization capabilities that outperform those of single networks. However, it has been recognized that for aggregation to be effective the individual networks must be as accurate and diverse as possible. An important problem is, then, how to tune the aggregate members in order to have an optimal compromise between these two conflicting conditions. We propose here a simple method for constructing regression/classification ensembles of neural networks that leads to overtrained aggregate members with an adequate balance between accuracy and diversity. The algorithm is favorably tested against other methods recently proposed in the literature, producing an improvement in performance on the standard statistical databases used as benchmarks. In addition, and as a concrete application, we apply our method to the sunspot time series and predict the remainder of the current cycle 23 of solar activity.