Solving these equations allows to simulate atmospheric conditions. However, this process demands substantial computational power, which is costly and non-trivial to maintain. Moreover, even on the most advanced high-performance computing (HPC) systems on earth, global weather forecasting simulations require multiple hours to conclude the computations.
This computational costs greatly hinders the improvements that can be made from increasing model resolution and ensemble sizes.
50Generative machine learning technology is leading to a paradigm shift in weather forecasting. Trained on heaps of ERA5 reanalysis data, generative AI has proven that it can replicate forecasts on a timescale similar to current numerical weather prediction forecasts but with much less computational effort, once the model is trained. This methodology is commonly referred to as surrogate modelling.
The exploration of data-driven global weather forecasting started half a decade ago with pioneering work at ECMWF. Yet, the true breakthrough came with the introduction of FourCastNet (Nvidia) in 2022, followed by contributions of Microsoft (ClimaX), Google-Deepmind (GraphCast), and lately Huawei (Pangu-Weather).
Each of these contributions has improved the state-of-the-art improving accuracy, model resolution, and the range of predicted physical variables. A recent statistical comparison, shows a good performance of the data-driven model (Pangu) in comparison with the operational traditional model employed at ECMWF (IFS), see also Figure 1. The model being skillfull for both upper-air variables, and surface variables (e.g. 2m temperature, 10m wind speed) when compared with observations. Moreover, great performance is observed also in the forecasting abilities of data-driven global weather models towards extreme events such as Tropical Cyclones.
Root mean square error (RMSE) scores of 500 hPa geopotential height for Pangu and IFS high-resolution forecasts over Europe for winter 2022/23 (6 days of forecast horizon), measured against operational analysis. Comparable accuracy is found for both Pangu and the IFS. Figure adopted from here.
The models described above, while powerful, don't fully adhere to the conservation equations underlying traditional numerical weather predictions. We anticipate that the next breakthrough will involve an AI model that accounts for conservation laws, as seen in Physics-Informed Neural Networks (PINNs). An introductory text on how PINNs can integrate fundamental physical laws with data-driven approaches is provided here.
Such a solution would hold various advantages:
A second key point where machine learning could provide a game-changing contribution comprises the data-assimilation process which combines observations with ongoing numerical weather predictions to provide initial boundary conditions.
The journey to enhance weather forecasting tools with AI modeling has been exhilarating. The excitement reached a new level when European Centre for Medium-Range Weather Forecasts unveiled their alpha version of the ML weather model, "AIFS" (Artificial Intelligence/Integrated Forecasting System). This marks a significant milestone in ECMWF's journey toward integrating machine learning throughout their operational forecasting system. Initial results are promising and currently under thorough evaluation.
At PropheSea, we believe that by combining software with mathematics, we can create a positive impact. Our applications are implemented across a broad range of industries and help our clients improve their processes.
Surrogate modelling within the industry has as key advantage that it provides a mean to substantially speed up the inference time of simulation models. This allows to evaluate more scenario's over the entire parameter space of interest, increasing confidence and allowing to optimize design studies. An illustrative industry example on how PropheSea introduced a surrogate model to create business value for DEME is provided here.
