Related Concept Videos
Prediction Intervals
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
Sensitivity, Specificity, and Predicted Value
Sensitivity is the...
Accuracy and Precision
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
Accuracy, limits, and approximation
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
Calibration Curves: Correlation Coefficient
You might also read
Related Articles
Articles linked to this work by shared authors, journal, and citation graph.
Population Dynamics of Perception and Emergence of Translational Membranes.
Metastable dynamical computing with energy landscapes: A primer.
Way More than the Sum of Their Parts: From Statistical to Structural Mixtures.
Related Experiment Video
Updated: Jun 28, 2025

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
Published on: October 11, 2018
Complexity-calibrated benchmarks for machine learning reveal when prediction algorithms succeed and mislead.
Sarah E Marzen1, Paul M Riechers2, James P Crutchfield3
1W. M. Keck Science Department of Pitzer, Scripps, and Claremont McKenna College, Claremont, CA, 91711, USA. smarzen@cmc.edu.
Next-generation reservoir computers, a type of recurrent neural network, struggle with complex time series prediction. New architectures are needed for optimal performance in forecasting financial and climate data.
Area of Science:
- Computational Neuroscience
- Machine Learning
- Time Series Analysis
Background:
- Recurrent neural networks (RNNs) are widely used for time series forecasting across various domains like finance, climate, and language.
- Reservoir computers are a simplified, easily trainable class of RNNs.
- A recent 'next-generation' reservoir computer model features a finite-past memory trace.
Purpose of the Study:
- To investigate the inherent limitations of finite-past memory traces in next-generation reservoir computers.
- To assess the performance of current RNNs in predicting complex, non-Markovian processes.
- To identify requirements for future RNN architectures.
Main Methods:
- Utilized Fano's inequality to establish a lower bound on prediction error for next-generation reservoir computers.
- Analyzed highly non-Markovian processes generated by large probabilistic state machines.
- Presented concentration-of-measure results for complex, randomly generated processes.
Main Results:
- Next-generation reservoir computers exhibit a prediction error significantly higher than the theoretical minimum for complex processes.
- Popular RNNs demonstrate suboptimal performance in predicting intricate time series.
- Large probabilistic state machines, particularly -machines, are crucial for generating unbiased training data.
Conclusions:
- Finite-past memory traces impose fundamental limitations on reservoir computer prediction accuracy.
- Current RNNs are not optimized for highly complex, non-Markovian time series.
- Development of novel, optimized RNN architectures is essential.
- Large probabilistic state machines serve as vital benchmarks for evaluating RNNs.

