Probabilistic Short-Term Sky Image Forecasting Using VQ-VAE and Transformer Models on Sky Camera Data.

Area of Science:

• Artificial Intelligence
• Renewable Energy Systems
• Atmospheric Science

Background:

• Cloud cover is a major factor affecting solar photovoltaic (PV) power output.
• Accurate short-term cloud movement prediction is crucial for reliable solar energy production planning.

Purpose of the Study:

• To develop a deep learning framework for direct cloud movement estimation from ground-based all-sky camera images.
• To improve the accuracy and reliability of solar energy forecasting by predicting cloud motion.

Main Methods:

• A three-step deep learning process: Convolutional Neural Network (CNN) for cloud segmentation, Vector Quantized Variational Autoencoder (VQ-VAE) for dimensionality reduction, and a GPT-style transformer for temporal prediction.
• Utilizing probabilistic cloud masks and discrete latent token sequences to capture cloud geometry and temporal dynamics.

Main Results:

• Achieved high accuracy in short-term predictions (IoU of 0.92, pixel accuracy of 0.96 at 5s ahead).
• Demonstrated robust performance in longer-term autoregressive predictions (IoU of 0.65, accuracy of 0.80 at 10 min).
• Developed a method for uncertainty estimation using token-level entropy, correlating with prediction error.

Conclusions:

• The proposed deep learning framework effectively estimates cloud movement from all-sky images.
• The system provides valuable uncertainty estimates for solar energy forecasting applications.
• Directly predicting cloud motion offers a promising alternative to predicting power output from historical data.