Loss functions in deep residual networks for short-term load forecasting: a systematic analysis

Area of Science:

• Electrical Engineering
• Data Science
• Artificial Intelligence

Background:

• Short-term load forecasting (STLF) is crucial for modern electricity systems.
• Deep Residual Networks (DRNs) show high predictive power for STLF.
• The impact of different loss functions on DRN performance for STLF is not well understood.

Purpose of the Study:

• To systematically evaluate various loss functions within original DRN and PCA-DRN frameworks for STLF.
• To analyze how loss functions affect forecasting performance and optimization behavior.
• To compare the performance of DRN and PCA-DRN with different loss functions on real-world datasets.

Main Methods:

• Evaluation of traditional, robust, and task-specific Penalized loss functions.
• Implementation within original DRN and Principal Component Analysis-Deep Residual Network (PCA-DRN) models.
• Testing on real-world datasets with diverse climatic and load characteristics.
• Statistical significance confirmed using bootstrap-based analysis.

Main Results:

• Charbonnier loss performed best overall in the original DRN framework.
• PCA-DRN showed performance divergence: Charbonnier for point-forecast error, Penalized for squared-error and correlation metrics.
• PCA-DRN consistently outperformed the original DRN, indicating benefits of dimensionality reduction.
• Statistical analysis confirmed the significance of improvements.

Conclusions:

• Loss function choice significantly impacts STLF accuracy and optimization.
• PCA-DRN enhances feature representation and generalization for STLF.
• Optimal loss function selection depends on data characteristics and feature representation for robust STLF.