Ross Bannister, November 2023
Data assimilation as a scientific tool for weather forecasting and beyond
In the early 2000s few academic groups around the world were doing research into the activity that we call “data assimilation”. Data assimilation is the process of combining imperfect model information and imperfect observations. This is extremely important for weather forecasting as without accurate and balanced starting conditions, forecast models cannot make meaningful predictions. Indeed all models need to know about “today’s weather” so they can propagate this up-to-the-minute information into a prediction of “tomorrow’s weather”, and beyond.
Knowing today’s weather is non-trivial. The atmosphere is an extensive fluid of around 5 billion cubic km and is described by multiple quantities (wind, temperature, pressure, humidity, cloud, etc.). It is turbulent and chaotic, especially when viewed at small scales. Despite the large number of observations, including from satellite, only a very small part of the volume is measured. This is why data assimilation is needed to combine the ‘theoretical’ model information with the real-world observations.
Data assimilation methods pioneered for weather forecasting are used for other systems too. These include the ocean, the hydrological cycle (e.g. for flood prediction), the carbon cycle, space weather, marine bio-chemical systems, and even disease spread (e.g. Covid). Furthermore, data assimilation is not just used to aid prediction. It is also used to produce datasets of these environmental systems for scientific analysis and societal/commercial exploitation.
Enter the DARC
DARC (the Data Assimilation Research Centre) started in the early 2000s under the directorship of Alan O’Neill, having its HQ in the Meteorology Department at the University of Reading (see the photo of the plaque on display there). It also involved scientists based in Oxford, Cambridge and Edinburgh Universities, and the Rutherford Appleton Laboratory. The initial focus was to use data assimilation to gain information about the stratosphere – a broad, stable layer of the atmosphere above the `weather’, accommodating the ozone layer.
Along with other centres around the UK, DARC was initiated with a budget and the prestigious status as a NERC (Natural Environmental Research Council) Centre of Excellence. Its initial remit was to progress the work of the DARE project (Data Assimilation in Readiness for EnviSat).
DARC scientists worked closely with the Met Office, using its brand new variational data assimilation system, to assimilate data from the EnviSat satellite. EnviSat was launched with great fanfare in 2002 and hosted instruments that could monitor important environmental quantities. DARC was concerned with assimilating stratospheric ozone, temperature and water vapour data from EnviSat (and other) instruments. Stratospheric water vapour, for instance, had never been assimilated before. In September 2002, the stratospheric polar vortex over the south pole – along with the ozone hole – split into two. Such a split in the southern hemisphere had never been seen before. EnviSat, and DARC, saw it happen.
There was also much interest in developing data assimilation methods (such as the variational method mentioned above). Early projects included the development of methods to extract the maximum amount of information from satellite observations, and using physical principles to better quantify errors in the models that observations are meant to reduce.
DARC has been led by several people, who have each encouraged DARC to grow in different directions. In the mid 2000s Martin Ehrendorfer became the director, then Peter Jan van Leeuwen, and then Alberto Carrassi. Since 2021 DARC has been led jointly between Sarah Dance and Amos Lawless. The DARC logo has changed over the years (see below).
Its status – as a NERC funded Centre of Excellence – remained until about 2008, when much of its remit was officially absorbed into NCEO (the National Centre for Earth Observation). DARC, however, remains as the identity of an active research group at the University of Reading.
DARC today
DARC is currently made up of about 25 scientists. Many still work on weather-related problems, but others work on a wider range of environmental systems, such as those mentioned above.
The variational method (much developed from the early days) is still the workhorse of weather forecasting, but DARC’s research also embraces ensemble, particle, and hybrid methods of solving theoretical and practical assimilation problems. DARC also runs an annual training course.
Incidentally, all of these data assimilation methods emerge from a fundamental theorem of probability called Bayes’ Theorem (reflecting the inevitable probabilistic approaches of dealing with uncertainty in complex systems), which DARC’s work has been faithful to from the very beginning.