By Guannan Hu
The importance of data assimilation
Data assimilation (DA) is a technique used to produce initial conditions for numerical weather prediction (NWP), where computer models describing the evolution of the atmosphere are used to predict future weather based on current or previous weather conditions. These models are usually very sensitive to initial conditions, meaning that slight changes in the initial conditions can lead to completely different weather forecasts. The Data Assimilation for the Resilient City (DARE) project is investigating the use of novel observations such as temperature data from vehicles, smartphone data, river camera images, etc. for weather and flooding forecasting. Accurate forecasting of hazardous weather events can help us to prepare in advance, and thus protecting lives and property and reducing economic losses.
DA is also used to create climate reanalyses, which are gridded datasets providing long-term historical estimates of climate variables covering the globe or a region. These datasets are used to monitor climate change.
The basic idea of data assimilation
Data assimilation blends observations with model forecasts to produce the best estimates of atmospheric and climate variables. For example, the air temperature on campus can be measured by a thermometer or predicted from past temperatures (and other relevant variables such as humidity and wind) using a computer model. Then we obtain the estimates for air temperature from two sources. We assume that the true temperature is somewhere in between. It can therefore be given by a weighted average of the two estimates, where the one with the smaller error has the greater weight as it is considered more reliable. This is a very simple example; the real data assimilation applications are much more complex and involve a huge amount of data.
The assimilation of novel observations
As computers become more powerful and the volume of observational data increases rapidly, data assimilation becomes increasingly important in improving the skills of weather forecasting. The assimilation of novel observations (e.g., geostationary satellite, radar data) has led to great improvement in forecast skill. Unlike thermometers and other conventional instruments, the weather satellite and Doppler radar measure the atmospheric variables indirectly. These observations need to be transformed for use in data assimilation procedures. This will cause so-called representation errors in addition to measurement errors. The geostationary satellite data and Doppler radar radial wind have been found to display strong spatial error correlations. In practical applications, these error correlations are usually taken into account indirectly in data assimilation systems or removed by thinning the observations. These approaches might be suboptimal as they prevent us from making full use of the observations. Accurately estimating observation error correlations for satellite and radar data can be very challenging. Satellite observations can have a mixture of inter-channel and spatial error correlations. Doppler radar radial wind has the error correlation lengthscales that may not be isotropic; they vary with the height of the observations and the distance of the observations to the radar. In addition, explicitly including correlated observation error statistics may largely increase the computational cost of DA. The increase is mainly caused by the inversion of dense matrices and the parallel communication costs in the computation of matrix-vector products. Another problem with including these error statistics is that it may change the convergence behaviour of the variational minimization procedure.
The more and more wide application of data assimilation
Starting with its use in the NWP, DA is now attracting more and more interest from the wider scientific community. People with different backgrounds and from different research institutes, universities, and weather services around the world are not only committed to developing new methods but are also keen to apply this technique to new areas. For instance, DA can be combined with machine learning. DA can be applied to space weather forecasting and even used to monitor and predict a pandemic!