A benchmark test of accuracy and precision in estimating dynamical systems characteristics from a time series

Characteristics of dynamical systems are often estimated to describe physiological processes. For instance, Lyapunov exponents have been determined to assess the stability of the cardio-vascular system, respiration, and, more recently, human gait and posture. However, the systematic evaluation of the accuracy and precision of these estimates is problematic because the proper values of the characteristics are typically unknown. We fill this void with a set of standardized time series with well-defined dynamical characteristics that serve as a benchmark. Estimates ought to match these characteristics, at least to good approximation. We outline a procedure to employ this generic benchmark test and illustrate its capacity by examining methods for estimating the maximum Lyapunov exponent. In particular, we discuss algorithms by Wolf and co-workers and by Rosenstein and co-workers and evaluate their performances as a function of signal length and signal-to-noise ratio. In all scenarios, the precision of Rosenstein's algorithm was found to be equal to or greater than Wolf's algorithm. The latter, however, appeared more accurate if reasonably large signal lengths are available and noise levels are sufficiently low. Due to its modularity, the presented benchmark test can be used to evaluate and tune any estimation method to perform optimally for arbitrary experimental data.