by Sarah L Dance, January 2024
Weather forecasts are an essential tool to help us prepare for hazardous weather events such as storms and floods. Early warnings are even more important in our changing climate as high-impact weather events are expected to become more frequent and intense. However, the production of forecasts has a large carbon footprint. For example, to initialize forecasts we must solve a large, computationally costly, mathematical problem known as data assimilation. Data assimilation involves 10s of millions of observations and 100s of millions of computational model variables. Supercomputers are used to solve these problems, and the energy costs are very large.
To address this issue, researchers are now investigating the potential of deep learning to significantly reduce the energy costs of forecast production. For example, since Autumn 2023, the European Centre for Medium-Range Weather Forecasts (ECMWF) have been running an AI version of their global forecast model running in routine experiments, known as AIFS.
Filling the skill gap
To achieve this ambition, there is a critical need for existing and future researchers to have skills and knowledge in modern data science. The UK Met Office is establishing a Transatlantic Data Science Academy, working in partnership with the US National Oceanic and Atmospheric Administration (NOAA). The Academy aims to address critical skills shortages in:
- data science
- data engineering
- data assimilation and
- research software engineering
Such skills increasingly underpin weather and climate science and the exploitation of Earth Observation data. A consortium of Universities from the Met Office Academic Partnership, led by the Data Assimilation Research Centre (DARC) at the University of Reading is helping to shape the Academy and provide a blueprint for its implementation.
The scoping project team held an online workshop in December 2023. Over 65 people registered to attend from around the world, including the UK, North and South America, China and Europe. The workshop generated some fantastic discussion and some great ideas for us to take forward. We will be publishing a first report with recommendations on how to establish an academy supporting a diverse group of scientists with tailored technical training, learning, personal and career development frameworks in March or April 2024.