Particle Filters for Flood Forecasting  (PFFF)

Particle Filters for Flood Forecasting  (PFFF)

A collaboration between: Dr Renaud Hostache, Luxembourg Institute of Science and Technology (LIST); Professors Nancy K Nichols and Peter Jan vanLeeuwen, University of Reading; Ms Concetta di Mauro, Luxembourg Institute of Science and Technology (LIST)

The objective  of this DARE pilot project is to investigate the application of advanced filters to assimilate high-resolution flood extent information derived from SAR images (75m spatial resolution) for the purposes of improving near real-time flood forecasts.  The forecasting system is composed of a hydrological model loosely coupled to a hydraulic model with uncertain rainfall forcing (from ERA interim).  The ensemble of model outputs is compared to satellite-derived flood probability maps taking into account satellite image classification uncertainty.  Standard ensemble Kalman filter (EnKF) methods that assume a normal distribution of the observation errors cannot be applied and therefore new filters need to be developed for the assimilation.  From experiments already carried out at LIST, three challenges arise:  (i) to prevent ensemble members/particles being given a weight of zero solely due to local mismatch at a few pixels;  (ii) to reduce biases due to over-prediction of flood extent (false positive) being penalized more strongly than under-prediction; and  (iii) to reduce the risk of particle degeneration, where weights for all but a few particles go to zero. The aim of the project will be to assess how these challenges can be met using new advanced filters that are being developed at the University of Reading, such as equal-weight particle filters and variational mapping particle filters.

Flood forecasting chains have been set up to enable the evaluation of the proposed data assimilation filters in controlled environments using synthetic (twin) experiments.  Two studies have been carried out using these systems.

  1. We first use a variant of a Particle Filter (PF), namely a PF with Sequential Importance Sampling (PF-SIS), to assimilate flood extent in near real-time into a hydrological hydraulic-model cascade. To reduce the risk of particle degeneration, a “tempering” power factor is applied to the conditional probability of the observation given the model prediction (also called likelihood in a PF). This allows inflation of the model posterior distribution variance. Various values of the “tempering coefficient”, leading to different Effective Ensemble Size (EES) are evaluated. The experiment shows that the assimilation framework yields correct results in the sense that the assimilation updates the particle weights so that the updated predictions move towards the synthetic truth. It also shows that the proposed tempering factor helps in reducing degeneracy while inflating posterior distribution variance. Fig. 1 shows the synthetic truth together with the ensemble expectations (ensemble weighted means) for the open loop (no assimilation) and the assimilation (using various tempering factor values) runs.  As shown in this figure, the experiment also demonstrates that the reduction of degeneracy is at the cost of a slight degradation of the overall performance as the higher the EES, the lower the performance of the assimilation run. This is shown by the black and blue lines moving closer to the synthetic truth (compared to orange and light blue line Figure 1).
  2. We also investigated how innovative satellite earth observations of soil moisture and flood extents can help in reducing errors and uncertainties in conceptual hydro-meteorological modelling, especially in ungauged areas where potentially no, or limited, runoff records are available. A spatially distributed conceptual hydrological model was developed to allow for the prediction of soil moisture and flood extents. Using rainfall and potential evapotranspiration time series derived from the globally and freely available ERA5 database as forcing of this model, long-term simulations of soil moisture, discharge and flood extents were carried out. Time series of soil moisture and flood extent observations derived from freely available satellite image databases were then jointly assimilated into the hydrological model in order to retrieve optimal parameter sets. The performance of the calibrated model was evaluated using the tempered PF in twin experiments. This synthetic experiment shows that the assimilation of long time series (~10 years) of observations of flood extents and soil moisture maps acquired every three days enable a satisfactory calibration of the hydrological model. The Nash Sutcliffe Efficiency, computed based on the comparison of simulated and synthetic discharge time series, reach high values (above 0.95) both during the calibration period and a 10-year validation period.

Figure 1: Water surface elevation time series at Saxons Lode: synthetic truth (red), open-loop (green) and assimilation experiments using the standard PF-SIS (black), and using various tempering factor values (blue, light blue and orange) enabling various effective ensemble sizes to be reached (indicated between parentheses as percentage of the ensemble size). The vertical dashed lines indicate the assimilation time steps. PF-SIS=Particle Filter with Sequential Importance Sampling. EES=Effective Ensemble Size.


Merging SAR-derived flood footprints with flood hazard maps for improved urban flood mapping

Merging SAR-derived flood footprints with flood hazard maps for improved urban flood mapping

Contributors: David Mason (University of Reading), John Bevington (JBA), Sarah Dance (University of Reading), Beatriz Revilla-Romero (JBA), Richard Smith (JBA), Sanita Vetra-Carvalho (University of Reading), Hannah Cloke (University of Reading).

This DARE pilot project is investigating a method of improving the accuracy of rapid post-event flood mapping in urban areas by merging pre-computed flood return period (FRP) maps with satellite synthetic aperture radar (SAR)-derived flood inundation maps. SAR sensors have the potential to detect flooding through cloud during both day- and night-time. The inputs are JBA’s Flood Foresight dynamic flood inundation extent and depth maps (updated every 3 hours), and a high resolution SAR image sequence. The SAR returns are used only in rural areas, including those adjacent to the urban areas, so that there is no need to take radar shadow and layover caused by buildings in urban areas into account. Also, rural SAR water level observations should be able to correct errors in model water elevations, because the JBA model thinks that all flooding is fluvial. On the other hand, it is an advantage to use the model’s FRP maps in urban areas, because these know where urban areas that are low are protected from flooding.

The project developed a method for detecting flooding in urban areas by merging near real-time SAR flood extents with model-derived FRP maps. The SAR flood detection is based on the fact that water generally appears dark in a SAR image. Urban areas that are protected (e.g. by embankments) have high return periods in the FRP maps, and their effective heights are correspondingly increased. The SAR water levels found in the rural areas are interpolated over the urban areas to take into account the fall-off of levels down the reach. The model waterline heights are similarly interpolated. The interpolated height maps from SAR and model are combined to a single map, which is used to estimate whether an urban pixel is flooded. The method was tested on urban flooding in West London in February 2014 (see image 3) and Tewkesbury in July 2007. It was compared to a previously-developed method that used SAR returns in both the rural and urban areas. The present method using SAR returns solely in rural areas gave an average flood detection accuracy of 94% and a false positive rate of 9% in the urban areas, and was more accurate than the previous method. A journal paper is in preparation.


Images: Urban flooding in West London in February 2014

Investigation of the ability of the renewed UK operational weather radar network to provide accurate real-time rainfall estimates for improved flood warnings.

Investigation of the ability of the renewed UK operational weather radar network to provide accurate real-time rainfall estimates for improved flood warnings.

by Dr RobThompson and Prof Anthony Illingworth, Dept of Meteorology, University of Reading

The UK operational radar network has the potential to deliver real-time rainfall estimates every five minutes with a resolution of 1km2 over most of the populated areas of the UK. If these rain rates were accurate then such data would have a major impact on the ability to predict short term ‘flash’ flooding events so that mitigating action can be taken. However, at present, for flood warnings, the accuracy is deemed insufficient, so the radar rainfall estimates from each radar are continuously adjusted using recent observations by ground-based rain gauges.

The UK radar network has recently been renewed and upgraded to dual polarisation, resulting in much improved data quality. In this study we will compare the radar signal obtained every five minutes from  the operational Dean Hill radar with the rain rate at the ground some 20km from the radar and just 400m below the radar beam and use this to validate and improve the retrieval algorithms that convert the radar return signal, or ‘reflectivity’ into a rain rate at the ground.

In the preparatory work  for this DARE pilot project we have been comparing the radar reflectivity observed every five minutes with the scanning Dean Hill radar 20km distant from Chilbolton where we have five different high resolution rain gauges. The radar pulse samples a volume 300m by 300m by 600m and is at a height of 400m above the gauges (see image).  One of these gauges measures the size distribution of the rain drops and so once we know the sizes and number of the drops we can calculate the radar reflectivity we would expect from the radar. Over the past two years we find close agreement, but there appears to be a slow drift in the radar calibration of about 60%. In collaboration with the Met Office we are trying to find the source of this drift.  However, once we correct for this drift, we find that with the new radar and its improved data quality there is a close correspondence between the rain from the radar and that observed at the ground.  This performance appears to be much better than previously obtained before the radars were upgraded.


Image. Multiple rain gauges

Investigating the uncertainty of weather radar data

Investigating the uncertainty of weather radar data

This blog describes the work of Masters student Vasiliki Kouroupaki, carried out in collaboration with the UK Met Office.

UK weather radar network

In numerical weather prediction, nowcast, hindcast and forecast models can be improved through data assimilation. Data assimilation is the technique which combines observations with output from a previous short-range forecast (background) to produce an optimal estimate of the state of the atmosphere (analysis).

Radar reflectivity observations are assimilated by the Met Office in order to provide up-to-date information about rainfall in the initial conditions for UK weather forecasts. In assimilation, observations are assigned weights according to their error statistics. Depending on the kind of observation, there are different factors or processes which can result in errors. In order to have an optimal analysis these errors must be correctly specified. However, due to the fact that the true errors are not known their statistics need to be estimated. In this work, the uncertainties of radar reflectivity observations assimilated into the Met Office UKV model are examined using a diagnostic technique. Data come from the operational UKV model with hourly cycling 4D-VAR or from trial experiments four times per day. The diagnostic is based on combinations of observation-minus- background, observation-minus-analysis and background-minus-analysis differences. The results show that observation error variances are higher for Winter (1 Dec 2017-18 Jan 2018) than for Summer (16 Jul-16 Aug 2018) and that they increase for higher reflectivity values. Further investigations classified the data by beam elevation and by radar ID. These showed that for values of beam elevation between 0.5- 1.0 and 3.0-4.0 degrees the error variance had greater values. Also, error statistics for different radars were positively correlated with the mean reflectivity observed by each radar.

Further investigation of observation error statistics in the assimilation could improve the initial conditions and thereby operational forecasts for convective rainfall events.