Earthformer: Exploring Space-Time Transformers for Earth System Forecasting

by   Zhihan Gao, et al.

Conventionally, Earth system (e.g., weather and climate) forecasting relies on numerical simulation with complex physical models and are hence both expensive in computation and demanding on domain expertise. With the explosive growth of the spatiotemporal Earth observation data in the past decade, data-driven models that apply Deep Learning (DL) are demonstrating impressive potential for various Earth system forecasting tasks. The Transformer as an emerging DL architecture, despite its broad success in other domains, has limited adoption in this area. In this paper, we propose Earthformer, a space-time Transformer for Earth system forecasting. Earthformer is based on a generic, flexible and efficient space-time attention block, named Cuboid Attention. The idea is to decompose the data into cuboids and apply cuboid-level self-attention in parallel. These cuboids are further connected with a collection of global vectors. We conduct experiments on the MovingMNIST dataset and a newly proposed chaotic N-body MNIST dataset to verify the effectiveness of cuboid attention and figure out the best design of Earthformer. Experiments on two real-world benchmarks about precipitation nowcasting and El Nino/Southern Oscillation (ENSO) forecasting show Earthformer achieves state-of-the-art performance.


page 2

page 9

page 21

page 22

page 23

page 24

page 25


PreDiff: Precipitation Nowcasting with Latent Diffusion Models

Earth system forecasting has traditionally relied on complex physical mo...

Simple Baseline for Weather Forecasting Using Spatiotemporal Context Aggregation Network

Traditional weather forecasting relies on domain expertise and computati...

AI Foundation Models for Weather and Climate: Applications, Design, and Implementation

Machine learning and deep learning methods have been widely explored in ...

A Novel Transformer Network with Shifted Window Cross-Attention for Spatiotemporal Weather Forecasting

Earth Observatory is a growing research area that can capitalize on the ...

TeleViT: Teleconnection-driven Transformers Improve Subseasonal to Seasonal Wildfire Forecasting

Wildfires are increasingly exacerbated as a result of climate change, ne...

MATNet: Multi-Level Fusion and Self-Attention Transformer-Based Model for Multivariate Multi-Step Day-Ahead PV Generation Forecasting

The integration of renewable energy sources (RES) into modern power syst...

Message in a Bottle – An Update to the Golden Record

Communication is an essential asset enabling humankind to forge an advan...

Please sign up or login with your details

Forgot password? Click here to reset