Developing reliable hourly electricity demand data through screening and imputation.

Area of Science:

• Energy Systems Analysis
• Data Science
• Electrical Engineering

Background:

• Electricity demand data are crucial for electric grid modeling, planning, and operation by utilities, governments, and academics.
• The U.S. Energy Information Administration collects hourly demand data from balancing authorities (BAs) nationwide.
• Significant data quality issues exist, with 2.2% missing and 0.5% anomalous values (outliers or negative) as of September 2019.

Purpose of the Study:

• To develop and validate a robust data cleaning process for electricity demand data.
• To ensure non-missing, continuous, and physically plausible demand data for improved grid analysis.
• To make cleaned, high-quality electricity demand data publicly available.

Main Methods:

• A screening process was developed to identify anomalous electricity demand values.
• Multiple Imputation by Chained Equations (MICE) was applied to impute missing and anomalous data points.
• Cross-validation was performed by artificially withholding data and using the MICE technique to predict it.

Main Results:

• The developed screening and imputation process successfully addressed missing and anomalous electricity demand data.
• Cross-validation demonstrated the accuracy of the imputation method, with a mean absolute percentage error of 3.5% across all balancing authorities.
• The cleaned dataset is published and available open access, facilitating further research and analysis.

Conclusions:

• The MICE technique provides an effective solution for imputing missing and anomalous electricity demand data.
• The availability of cleaned, high-quality data is essential for accurate electric grid modeling and planning.
• This work contributes to improving the reliability and usability of national electricity demand datasets.