Earthly Machine Learning

🌍 Abstract:Projecting climate change is a generalization problem: we extrapolate the recent past using physical models across past, present, and future climates. Current climate models require representations of processes that occur at scales smaller than the model grid size, which remain the main source of projection uncertainty. Recent machine learning (ML) algorithms offer promise for improving these process representations but often extrapolate poorly outside their training climates. To bridge this gap, the authors propose a “climate-invariant” ML framework, incorporating knowledge of climate processes into ML algorithms, and show that this approach enhances generalization across different climate regimes.📌 Key Points:Highlights how ML models in climate science struggle to generalize beyond their training data, limiting their utility in future climate projections.Introduces a "climate-invariant" ML framework, embedding physical climate process knowledge into ML models through feature transformations of input and output data.Demonstrates that neural networks with climate-invariant design generalize better across diverse climate conditions in three atmospheric models, outperforming raw-data ML approaches.Utilizes explainable AI methods to show that climate-informed mappings learned by neural networks are more spatially local, improving both interpretability and data efficiency.💡 The Big Idea:Combining machine learning with physical insights through a climate-invariant approach enables models that not only learn from data but also respect the underlying physics—paving the way for more reliable and generalizable climate projections.📖 Citation:Beucler, Tom, et al. "Climate-invariant machine learning." Science Advances 10.6 (2024): eadj7250. DOI: 10.1126/sciadv.adj7250

🎙️ Episode 25: ClimaX: A foundation model for weather and climateDOI: https://doi.org/10.48550/arXiv.2301.10343🌀 Abstract:Most cutting-edge approaches for weather and climate modeling rely on physics-informed numerical models to simulate the atmosphere's complex dynamics. These methods, while accurate, are often computationally demanding, especially at high spatial and temporal resolutions. In contrast, recent machine learning methods seek to learn data-driven mappings directly from curated climate datasets but often lack flexibility and generalization. ClimaX introduces a versatile and generalizable deep learning model for weather and climate science, capable of learning from diverse, heterogeneous datasets that cover various variables, time spans, and physical contexts.📌 Bullet points summary:ClimaX is a flexible foundation model for weather and climate, overcoming the rigidity of physics-based models and the narrow focus of traditional ML approaches by training on heterogeneous datasets.The model utilizes Transformer-based architecture with novel variable tokenization and aggregation mechanisms, allowing it to handle diverse climate data efficiently.Pre-trained via a self-supervised randomized forecasting objective on CMIP6-derived datasets, ClimaX learns intricate inter-variable relationships, enhancing its adaptability to various forecasting tasks.Demonstrates strong, often state-of-the-art performance across tasks like multi-scale weather forecasting, climate projections (ClimateBench), and downscaling — sometimes outperforming even operational systems like IFS.The study highlights ClimaX's scalability, showing performance gains with more pretraining data and higher resolutions, underscoring its potential for future developments with increased data and compute resources.💡 Big idea:ClimaX represents a shift toward foundation models in climate science, offering a single, adaptable architecture capable of generalizing across a wide array of weather and climate modeling tasks — setting the stage for more efficient, data-driven climate research.📖 Citation:Nguyen, Tung, et al. "Climax: A foundation model for weather and climate." arXiv preprint arXiv:2301.10343 (2023).

🎙️ Episode 24: AI-empowered Next-Generation Multiscale Climate Modelling for Mitigation and Adaptation🔗 DOI: https://doi.org/10.1038/s41561-024-01527-w🌐 AbstractDespite decades of progress, Earth system models (ESMs) still face significant gaps in accuracy and uncertainty, largely due to challenges in representing small-scale or poorly understood processes. This episode explores a transformative vision for next-generation climate modeling—one that embeds AI across multiple scales to enhance resolution, improve model fidelity, and better inform climate mitigation and adaptation strategies.📌 Bullet points summaryExisting ESMs struggle with inaccuracies in climate projections due to subgrid-scale and unknown process limitations.A new approach is proposed that blends AI with multiscale modeling, combining fine-resolution simulations with coarser hybrid models that capture key Earth system feedbacks.This strategy is built on four pillars:Higher resolution via advanced computingPhysics-aware machine learning to enhance hybrid modelsSystematic use of Earth observations to constrain modelsModernized scientific infrastructure to operationalize insightsAims to deliver faster, more actionable climate data to support urgent policy needs for both mitigation and adaptation.Envisions hybrid ESMs and interactive Earth digital twins, where AI helps simulate processes more realistically and supports climate decision-making at scale.💡 The Big IdeaIntegrating AI into climate models across scales is not just an upgrade—it’s a shift towards smarter, faster, and more adaptive climate science, essential for responding to the climate crisis with precision and urgency.📖 CitationEyring, Veronika, et al. "AI-empowered next-generation multiscale climate modelling for mitigation and adaptation." Nature Geoscience 17.10 (2024): 963–971.

🎙️ Episode 23: FourCastNet – Accelerating Global High-Resolution Weather Forecasting Using Adaptive Fourier Neural Operators🔗 DOI: https://doi.org/10.1145/3592979.3593412🌍 AbstractAs climate change intensifies extreme weather events, traditional numerical weather prediction (NWP) struggles to keep pace due to computational limits. This episode explores FourCastNet, a deep learning Earth system emulator that delivers high-resolution, medium-range global forecasts at unprecedented speed—up to five orders of magnitude faster than NWP—while maintaining near state-of-the-art accuracy.📌 Bullet points summaryFourCastNet outpaces traditional NWP with forecasts that are not only faster by several magnitudes but also comparably accurate, thanks to its data-driven deep learning approach.Powered by Adaptive Fourier Neural Operators (AFNO), the model efficiently handles high-resolution data, leveraging spectral convolutions, model/data parallelism, and performance optimizations like CUDA graphs and JIT compilation.Scales excellently across supercomputers such as Selene, Perlmutter, and JUWELS Booster, reaching 140.8 petaFLOPS and enabling rapid training and large-scale ensemble forecasts.Addresses long-standing challenges in weather and climate modeling, including limits in resolution, complexity, and throughput, paving the way for emulating fine-scale Earth system processes.Enables "Interactivity at Scale"—supporting digital Earth twins and empowering users to explore future climate scenarios interactively, aiding science, policy, and public understanding.💡 The Big IdeaFourCastNet revolutionizes weather forecasting by merging the power of deep learning and spectral methods, unlocking interactive, ultra-fast, and high-fidelity Earth system simulations for a changing world.📖 CitationKurth, Thorsten, et al. "Fourcastnet: Accelerating global high-resolution weather forecasting using adaptive fourier neural operators." Proceedings of the Platform for Advanced Scientific Computing Conference. 2023.

🎙️ Episode 22: Knowledge-guided machine learning can improve carbon cycle quantification in agroecosystems🔗 DOI: https://doi.org/10.1038/s41467-023-43860-5🧠 AbstractImproving the accuracy and scalability of carbon cycle quantification in agroecosystems is essential for climate mitigation and sustainable agriculture. This episode discusses a new Knowledge-Guided Machine Learning (KGML) framework that integrates process-based models, high-resolution remote sensing, and machine learning to address key limitations in conventional approaches.📌 Bullet points summaryIntroduces KGML-ag-Carbon, a hybrid model combining process-based simulation (ecosys), remote sensing, and ML to improve carbon cycle modeling in agroecosystems.Outperforms traditional models in capturing spatial and temporal carbon dynamics across the U.S. Corn Belt, especially under data-scarce conditions.Delivers high-resolution (250m daily) estimates for critical carbon metrics such as GPP, Ra, Rh, NEE, and crop yield, with field-level precision.Benefits from pre-training with synthetic data, remote sensing assimilation, and a hierarchical architecture with knowledge-guided loss functions for better accuracy and interpretability.Shows promise for broader applications including nutrient cycle modeling, large-scale carbon assessment, and scenario testing under various management and climate conditions.💡 The Big IdeaKGML-ag-Carbon represents a leap in modeling agroecosystem carbon cycles, blending scientific knowledge with data-driven insights to unlock precision and scalability in climate-smart agriculture.📖 CitationLiu, Licheng, et al. "Knowledge-guided machine learning can improve carbon cycle quantification in agroecosystems." Nature Communications 15.1 (2024): 357.

🎙️ Episode 21 — AtmoRep: A Stochastic Model of Atmospheric Dynamics Using Large-Scale Representation LearningThis week, we explore AtmoRep, a novel task-independent AI model for simulating atmospheric dynamics. Built on large-scale representation learning and trained on ERA5 reanalysis data, AtmoRep delivers strong performance across a variety of tasks—without needing task-specific training.🔍 Highlights from the episode:Introduction to AtmoRep, a stochastic computer model leveraging AI to simulate the atmosphere.Zero-shot capabilities for nowcasting, temporal interpolation, model correction, and generating counterfactuals.Outperforms or matches state-of-the-art models like Pangu-Weather and even ECMWF's IFS at short forecast horizons.Fine-tuning with additional data, like radar observations, enhances performance—especially for precipitation forecasts.Offers a computationally efficient alternative to traditional numerical models, with potential for broader scientific and societal applications.📚 Read the paper: https://doi.org/10.48550/arXiv.2308.13280✍️ Citation:Lessig, Christian, et al. "AtmoRep: A stochastic model of atmosphere dynamics using large scale representation learning." arXiv:2308.13280 (2023)

🎙️ Episode 20: Finding the Right XAI Method—Evaluating Explainable AI in Climate Science🔗 DOI: https://doi.org/10.48550/arXiv.2303.00652🧩 AbstractExplainable AI (XAI) methods are increasingly used in climate science, but the lack of ground truth explanations makes it difficult to evaluate and compare them effectively. This episode dives into a new framework for systematically evaluating XAI methods based on key properties tailored to climate research needs.📌 Bullet points summaryIntroduces XAI evaluation for climate science, offering a structured approach to assess and compare explanation methods using key desirable properties.Identifies five critical properties for XAI in this context: robustness, faithfulness, randomization, complexity, and localization.Evaluation shows that different XAI methods perform differently across these properties, with performance also depending on model architecture.Salience methods often score well on faithfulness and complexity but lower on randomization.Sensitivity methods typically do better on randomization but at the expense of other properties.Proposes a framework to guide method selection: assess the importance of each property for the research task, compute skill scores for available methods, and rank or combine methods accordingly.Highlights the role of benchmark datasets and evaluation metrics in supporting transparent and context-specific XAI adoption in climate science.💡 The Big IdeaThis work empowers climate researchers to make informed, task-specific choices in explainable AI, turning a fragmented XAI landscape into a guided, comparative process rooted in scientific needs.📖 CitationBommer, Philine Lou, et al. "Finding the right XAI method—A guide for the evaluation and ranking of explainable AI methods in climate science." Artificial Intelligence for the Earth Systems 3.3 (2024): e230074.

🎧 Abstract:Weather forecasting is essential for both science and society. This episode explores a breakthrough in medium-range global weather forecasting using artificial intelligence. The researchers introduce Pangu-Weather, an AI-powered system that leverages 3D deep networks with Earth-specific priors and a hierarchical temporal aggregation strategy to significantly enhance forecast accuracy and reduce error accumulation over time.📌 Bullet points summary:Pangu-Weather applies 3D deep learning with Earth-specific priors for accurate medium-range global weather forecasts.It utilizes a hierarchical temporal aggregation strategy to minimize accumulation errors.Outperforms ECMWF’s operational Integrated Forecasting System (IFS) in deterministic forecasting and tropical cyclone tracking.Achieves over 10,000× faster performance than IFS, enabling efficient large-member ensemble forecasts.Though trained on reanalysis data and limited in variable scope, Pangu-Weather presents a promising hybrid approach combining AI and traditional numerical weather prediction (NWP).💡 The Big Idea:AI is reshaping how we predict the weather. With Pangu-Weather, deep learning meets atmospheric science—delivering faster, more accurate forecasts that could redefine the future of meteorology.📚 Citation:Bi, K., Xie, L., Zhang, H. et al. Accurate medium-range global weather forecasting with 3D neural networks. Nature 619, 533–538 (2023). https://doi.org/10.1038/s41586-023-06185-3

🎧 Abstract:In this episode, we dive into GraphDOP, a novel data-driven forecasting system developed by ECMWF. Unlike traditional models, GraphDOP learns directly from Earth System observations—without relying on physics-based reanalysis. By capturing relationships between satellite and conventional observations, it builds a latent representation of Earth’s dynamic systems and delivers accurate weather forecasts up to five days ahead.📌 Bullet points summary:GraphDOP is developed by ECMWF and operates purely on observational data, without physics-based (re)analysis or feedback.Produces skillful forecasts for surface and upper-air parameters up to five days into the future.Competes with ECMWF’s IFS for two-metre temperature (t2m), outperforming it in the Tropics at 5-day lead times.Can generate forecasts at any time and location—even where observational data is sparse—without using gridded ERA5 fields for training.Combines data from various instruments to create accurate joint forecasts of surface and tropospheric temperatures in the Tropics.Learns observation relationships that generalize well to data-sparse regions, with upper-level wind forecasts aligning closely with ERA5 even in low-coverage areas.💡 The Big Idea:GraphDOP reimagines weather forecasting by proving that pure observational data—when paired with intelligent modeling—can rival and even surpass traditional, physics-based systems in both speed and accuracy.📚 Citation:Alexe, Mihai, et al. "GraphDOP: Towards skilful data-driven medium-range weather forecasts learnt and initialised directly from observations." arXiv preprint arXiv:2412.15687 (2024). https://doi.org/10.48550/arXiv.2412.15687

🎧 Abstract:In this episode, we explore DiffDA, a novel data assimilation approach for weather forecasting and climate modeling. Built on the foundations of denoising diffusion models, DiffDA uses the pretrained GraphCast neural network to assimilate atmospheric variables from predicted states and sparse observations—providing a data-driven pathway to generate accurate initial conditions for forecasts.📌 Bullet points summary:Introduces DiffDA, a machine learning-based data assimilation method that leverages predicted states and sparse observations.Utilizes the pretrained GraphCast weather model, repurposed as a denoising diffusion model.Employs a two-phase conditioning strategy: on predicted states (training/inference) and sparse observations (inference only).Capable of generating assimilated global atmospheric data at 0.25° resolution.Demonstrates that initial conditions created via DiffDA retain forecast quality with a lead time degradation of at most 24 hours compared to top-tier assimilation systems.Enables autoregressive reanalysis dataset generation without full observation availability.💡 The Big Idea:DiffDA represents a step forward in data assimilation—merging the strengths of diffusion models and machine learning to produce accurate, observation-consistent initial conditions for future-focused forecasting.📚 Citation:Huang, Langwen, et al. "Diffda: a diffusion model for weather-scale data assimilation." arXiv preprint arXiv:2401.05932 (2024). https://doi.org/10.48550/arXiv.2401.059327