Search research articles

Related Concept Videos

Survival Tree

Survival Tree

Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...

Introduction to R

Introduction to R

R is a powerful software environment for statistical computing and graphics. Originating as an implementation of the S language, developed at Bell Laboratories, R has evolved into a robust, open-source statistical software favored by statisticians and data scientists worldwide. Its comprehensive suite includes data manipulation, calculation, and graphical display capabilities, making it versatile for data analysis and visualization. Its programming language is at the core of R's...

Steps in Outbreak Investigation

Steps in Outbreak Investigation

In the ever-evolving field of public health, statistical analysis serves as a cornerstone for understanding and managing disease outbreaks. By leveraging various statistical tools, health professionals can predict potential outbreaks, analyze ongoing situations, and devise effective responses to mitigate impact. For that to happen, there are a few possible stages of the analysis:

Random Error

Random Error

Random or indeterminate errors originate from various uncontrollable variables, such as variations in environmental conditions, instrument imperfections, or the inherent variability of the phenomena being measured. Usually, these errors cannot be predicted, estimated, or characterized because their direction and magnitude often vary in magnitude and direction even during consecutive measurements. As a result, they are difficult to eliminate. However, the aggregate effect of these errors can be...

Statistical Software for Data Analysis and Clinical Trials

Statistical Software for Data Analysis and Clinical Trials

Statistical software is pivotal in data analysis and clinical trials by providing tools to analyze data, draw conclusions, and make predictions. These software packages range from simple data management applications to complex analytical platforms, supporting various statistical tests, models, and simulation techniques. Their significance lies in their ability to handle vast amounts of data with precision and efficiency, enabling researchers to validate hypotheses, identify trends, and make...

What are Estimates?

What are Estimates?

It isn't easy to measure a parameter such as the mean height or the mean weight of a population. So, we draw samples from the population and calculate the mean height or mean weight of the individuals in the sample. This sample data acts as a representative measure of the population parameter. These sample statistics are known as estimates.
The estimate for the mean of a sample is denoted by ͞x, whereas the mean of the population is designated as μ. Further, parameters such...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Refined trajectory smoothing and deep learning classification of human sperm motility.

Human reproduction (Oxford, England)·2026

Same author

An Artificial Intelligence-Based Computer Vision Model for Human Sperm Concentration, Motility, and Kinematics Analysis.

Smart medicine·2026

Same author

The fattening speed: Understanding the impact of internet speed on obesity, and the mediating role of sedentary behaviour.

Economics and human biology·2024

Same author

Internet and Gambling: Insights from Australia's NBN Rollout.

Journal of gambling studies·2024

Same author

Probabilistic Causal Effect Estimation With Global Neural Network Forecasting Models.

IEEE transactions on neural networks and learning systems·2022

Same author

LSTM-MSNet: Leveraging Forecasts on Sets of Related Time Series With Multiple Seasonal Patterns.

IEEE transactions on neural networks and learning systems·2020

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Search research articles

Related Experiment Video

Updated: Aug 17, 2025

Design and Analysis for Fall Detection System Simplification

Design and Analysis for Fall Detection System Simplification

Published on: April 6, 2020

Forecast evaluation for data scientists: common pitfalls and best practices.

Hansika Hewamalage¹, Klaus Ackermann², Christoph Bergmeir³

¹School of Computer Science & Engineering, University of New South Wales, Sydney, Australia.

Data Mining and Knowledge Discovery

|December 12, 2022

Summary

Machine Learning (ML) and Deep Learning (DL) show promise in time series forecasting but struggle with non-stationarities. This work bridges the knowledge gap by detailing forecast evaluation best practices for ML researchers.

Keywords:

Forecast evaluation Time series forecasting

More Related Videos

Watershed Planning within a Quantitative Scenario Analysis Framework

Watershed Planning within a Quantitative Scenario Analysis Framework

Published on: July 24, 2016

An R-Based Landscape Validation of a Competing Risk Model

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

Related Experiment Videos

Last Updated: Aug 17, 2025

Design and Analysis for Fall Detection System Simplification

Design and Analysis for Fall Detection System Simplification

Published on: April 6, 2020

Watershed Planning within a Quantitative Scenario Analysis Framework

Watershed Planning within a Quantitative Scenario Analysis Framework

Published on: July 24, 2016

An R-Based Landscape Validation of a Competing Risk Model

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

Area of Science:

Data Science
Machine Learning
Statistics

Background:

Machine Learning (ML) and Deep Learning (DL) show increasing competitiveness in time series forecasting with large datasets.
Non-stationarities in time series data pose significant challenges for data-driven ML models.
Forecast evaluation methodologies are not widely understood within the ML community, leading to potential misinterpretations of model performance.

Purpose of the Study:

To provide a tutorial-like compilation of forecast evaluation details tailored for ML researchers.
To bridge the knowledge gap between traditional forecasting methods and state-of-the-art ML techniques.
To address flawed evaluation practices in ML that can lead to spurious conclusions about model competitiveness.

Main Methods:

Elaboration on problematic time series characteristics like non-normality and non-stationarities.
Outline of best practices in forecast evaluation, including data partitioning, error calculation, and statistical testing.
Guidelines for selecting appropriate error measures based on dataset characteristics.

Main Results:

Identification of common pitfalls in forecast evaluation stemming from time series properties.
Demonstration of how flawed evaluation practices can lead to misleading conclusions about ML model performance.
A structured approach to forecast evaluation for ML practitioners.

Conclusions:

Accurate forecast evaluation is crucial for the reliable application of ML and DL in time series forecasting.
Understanding time series characteristics and adopting rigorous evaluation practices are essential for ML researchers.
This work aims to improve the robustness and interpretability of ML-based forecasting models.