WxC-Bench: A Novel Dataset for Weather and Climate Downstream Tasks

High-quality openly-accessible machine learning (ML)-ready datasets play a foundational role in developing new artificial intelligence (AI) models or fine-tuning existing models for scientific applications such as weather and climate analysis. However, despite the growing development of new deep learning models for weather and climate, there is a scarcity of curated, pre-processed ML-ready datasets. Curating such high-quality datasets for developing new models is challenging particularly because the modality of the input data varies significantly for different downstream tasks addressing different atmospheric scales (spatial and temporal). Here we introduce WxC-Bench (Weather and Climate Bench), a multi-modal dataset designed to support the development of generalizable AI models for various downstream use-cases in weather and climate research. WxC-Bench supports examining several atmospheric processes from meso-β (20 - 200 km) scale to synoptic scales (2500 km), such as aviation turbulence, hurricane intensity and track monitoring, weather analog search, gravity wave parameterization, and natural language report generation. We provide a comprehensive description of the dataset and also present a technical validation for baseline analysis. The dataset and code to prepare the ML-ready data have been made publicly available on Hugging Face, and can be accessed using WxC-Bench Python package.