LMcast: A pretrained language model guided long-term memory transformer for precipitation nowcasting.

Area of Science:

• Meteorology and Atmospheric Sciences
• Artificial Intelligence
• Machine Learning

Background:

• Deep learning models show promise in precipitation nowcasting but struggle with the inherent uncertainty and chaotic evolution of rainfall systems.
• Existing methods often focus on immediate motion trends or local details, facing challenges in capturing long-term rainfall system dynamics as lead time increases.
• Limited effective information in longer lead times hinders accurate prediction of future rainfall patterns.

Purpose of the Study:

• To address the challenge of insufficient effective information in precipitation nowcasting for longer lead times.
• To leverage the retrieval and generation capabilities of large language models (LLMs) for improved rainfall prediction.
• To introduce a novel precipitation nowcasting model, LMcast, that utilizes historical data as prior knowledge.

Main Methods:

• Proposed LMcast, a precipitation nowcasting model incorporating a long-term memory recall mechanism guided by pre-trained language models.
• Utilized the linguistic knowledge of LLMs to retrieve historical rainfall data from a codebook, serving as prior information.
• Designed a fusion architecture to integrate recalled historical data (long-term memory) with current input data (short-term memory) for final predictions.

Main Results:

• LMcast demonstrated effectiveness in precipitation nowcasting tasks.
• The model showed superiority compared to state-of-the-art techniques across four publicly available radar datasets.
• The long-term memory recall approach effectively resolved the issue of insufficient information for extended lead times.

Conclusions:

• LMcast successfully integrates LLM capabilities for enhanced precipitation nowcasting.
• The proposed model offers a significant improvement over existing methods, particularly for predicting long-term rainfall trends.
• The fusion architecture effectively combines historical context with real-time data for accurate predictions.