A new chapter begins!

by Guannan Hu

I am glad to have joined the DARE project team and to start working at the Department of Meteorology at the University of Reading. The working environment is fantastic. People are very friendly. They helped me settle in well and always make me feel welcome.

I completed my PhD at the University of Hamburg last year. My research areas were data assimilation and extreme value theory. I attempted to find general ways that can improve the performance of data assimilation. I also investigated whether data assimilation can be accurate in predicting extreme events. I wanted to figure out what the main factors are that prevent us from achieving accurate predictions. Is it observation, model forecast, or data assimilation algorithm?

From my BSc to PhD, I have always been engaged in meteorology. I could not image this at the very beginning as I thought that the people studying meteorology would be weather broadcasters on television later. My journey continues, and I am ready for the new chapter! Image taken from the meteorology institute on the University of Hamburg.

Data-assimilation of crowdsourced weather stations for urban heat and water studies in Birmingham as a testbed

Data-assimilation of crowdsourced weather stations for urban heat and water studies in Birmingham as a testbed

DARE pilot project with participants:  Professor Lee Chapman, University of Birmingham;  Sytse Koopmans; Gert-Jan Steeneveld, Wageningen University & Research (WUR)

The research will be a verification by ground truth observations of the Birmingham Urban Climate Lab (BUCL) observational network.

The worktask aims are:

  • Setting up basic model run with WRF with meteorological boundaries from ECMWF
  • Remapping land use from CORINE land cover inventory (100 m) to USGS classification used by WRF
  • Calculate land use fraction by Landsat 8 satellite
  • Derived urban morphology indicators with the NUDAPT tool

For the project we completed the model infrastructure without data assimilation and a basic run without was conducted accordingly. The main effort done in the last year was to create a geographical dataset that satisfy the 200 meter resolution. Before applying data assimilation it is important to describe the land use and urban characteristics as well as possible.

  • The default available land use datasets were too course for a 200m run. We have remapped the 100m Corine land use dataset to a format which can be read by WRF.
  • The urban morphology has been improved by applying the National Urban Database and Access Portal Tool (NUDAPT) (Ching et al, 2009*). With this step we progressed the representation of urban morphology in the WRF model compared to previous studies.

NUDAPT has been tested against more elementary and basic representations in WRF in a study for a Chinese city. The performance of NUDAPT looks promising.

Image 1 – Mean building height Birmingham

 

Image 2: Land use fraction Birmingham

*Ching J, Brown M, McPherson T, Burian S, Chen F, Cionco R, Hanna A, Hultgren T, Sailor D, Taha H, Williams D. 2009. National Urban Database and Access Portal Tool, NUDAPT. Bulletin of the American Meteorological Society 90( 8): 1157– 1168.

Particle Filters for Flood Forecasting  (PFFF)

Particle Filters for Flood Forecasting  (PFFF)

A collaboration between: Dr Renaud Hostache, Luxembourg Institute of Science and Technology (LIST); Professors Nancy K Nichols and Peter Jan vanLeeuwen, University of Reading; Ms Concetta di Mauro, Luxembourg Institute of Science and Technology (LIST)

The objective  of this DARE pilot project is to investigate the application of advanced filters to assimilate high-resolution flood extent information derived from SAR images (75m spatial resolution) for the purposes of improving near real-time flood forecasts.  The forecasting system is composed of a hydrological model loosely coupled to a hydraulic model with uncertain rainfall forcing (from ERA interim).  The ensemble of model outputs is compared to satellite-derived flood probability maps taking into account satellite image classification uncertainty.  Standard ensemble Kalman filter (EnKF) methods that assume a normal distribution of the observation errors cannot be applied and therefore new filters need to be developed for the assimilation.  From experiments already carried out at LIST, three challenges arise:  (i) to prevent ensemble members/particles being given a weight of zero solely due to local mismatch at a few pixels;  (ii) to reduce biases due to over-prediction of flood extent (false positive) being penalized more strongly than under-prediction; and  (iii) to reduce the risk of particle degeneration, where weights for all but a few particles go to zero. The aim of the project will be to assess how these challenges can be met using new advanced filters that are being developed at the University of Reading, such as equal-weight particle filters and variational mapping particle filters.

Flood forecasting chains have been set up to enable the evaluation of the proposed data assimilation filters in controlled environments using synthetic (twin) experiments.  Two studies have been carried out using these systems.

  1. We first use a variant of a Particle Filter (PF), namely a PF with Sequential Importance Sampling (PF-SIS), to assimilate flood extent in near real-time into a hydrological hydraulic-model cascade. To reduce the risk of particle degeneration, a “tempering” power factor is applied to the conditional probability of the observation given the model prediction (also called likelihood in a PF). This allows inflation of the model posterior distribution variance. Various values of the “tempering coefficient”, leading to different Effective Ensemble Size (EES) are evaluated. The experiment shows that the assimilation framework yields correct results in the sense that the assimilation updates the particle weights so that the updated predictions move towards the synthetic truth. It also shows that the proposed tempering factor helps in reducing degeneracy while inflating posterior distribution variance. Fig. 1 shows the synthetic truth together with the ensemble expectations (ensemble weighted means) for the open loop (no assimilation) and the assimilation (using various tempering factor values) runs.  As shown in this figure, the experiment also demonstrates that the reduction of degeneracy is at the cost of a slight degradation of the overall performance as the higher the EES, the lower the performance of the assimilation run. This is shown by the black and blue lines moving closer to the synthetic truth (compared to orange and light blue line Figure 1).
  2. We also investigated how innovative satellite earth observations of soil moisture and flood extents can help in reducing errors and uncertainties in conceptual hydro-meteorological modelling, especially in ungauged areas where potentially no, or limited, runoff records are available. A spatially distributed conceptual hydrological model was developed to allow for the prediction of soil moisture and flood extents. Using rainfall and potential evapotranspiration time series derived from the globally and freely available ERA5 database as forcing of this model, long-term simulations of soil moisture, discharge and flood extents were carried out. Time series of soil moisture and flood extent observations derived from freely available satellite image databases were then jointly assimilated into the hydrological model in order to retrieve optimal parameter sets. The performance of the calibrated model was evaluated using the tempered PF in twin experiments. This synthetic experiment shows that the assimilation of long time series (~10 years) of observations of flood extents and soil moisture maps acquired every three days enable a satisfactory calibration of the hydrological model. The Nash Sutcliffe Efficiency, computed based on the comparison of simulated and synthetic discharge time series, reach high values (above 0.95) both during the calibration period and a 10-year validation period.

Figure 1: Water surface elevation time series at Saxons Lode: synthetic truth (red), open-loop (green) and assimilation experiments using the standard PF-SIS (black), and using various tempering factor values (blue, light blue and orange) enabling various effective ensemble sizes to be reached (indicated between parentheses as percentage of the ensemble size). The vertical dashed lines indicate the assimilation time steps. PF-SIS=Particle Filter with Sequential Importance Sampling. EES=Effective Ensemble Size.

 

Controlling and mitigating urban flooding with DA

Controlling and mitigating urban flooding with DA

by Prof Onno Bokhove and Tom Kent, PDRA,  University of Leeds
(The University of Leeds is a collaborator on the DARE project).

Motivated by the Boxing Day 2015 floods in Yorkshire (involving the Aire and Calder Rivers), we aim (i) to explore strategies of dynamic flood control and mitigation, and (ii) to assess and communicate flood-mitigation schemes in a concise and straightforward manner in order to assist decision-making for policy makers and inform the general public. To achieve our objectives, we are developing idealised observing system simulation experiments (OSSEs) using novel numerical models based on the Wetropolis flood demonstrator. Wetropolis is a physical model that provides a scientific testing environment for flood modelling, control and mitigation, and data assimilation, and has inspired numerous discussions with flood practitioners, policy makers and the public. Such discussions led us to revisit and refine a procedure that offers both a complementary diagnostic for classifying flood events (from gauge data and/or simulations) and a protocol to optimise the assessment of mitigation schemes via comprehensible cost-effectiveness analyses.

We have developed a protocol that revisits the concept of flood-excess volume (FEV). It is often difficult to grasp how much water is responsible for the damage caused by an extreme flood event, and how much of this floodwater can be mitigated by certain mitigation measures. Our protocol not only quantifies the magnitude of a flood but also establishes the cost-effectiveness of a suite of ‘grey’ engineering-based measures and ‘green’ nature-based solutions. Using river-level gauge data and mitigation schemes from the UK and French rivers, we demonstrate objectively the effectiveness of measures that can help stakeholders make decisions based on both technical and environmental criteria. The protocol should form a preliminary analysis, to be conducted prior to more detailed hydraulic modelling studies. In collaboration with colleagues from Univ. Grenoble, our work has been published in an international journal and further disseminated at numerous meetings and conferences. To date, it has contributed to the EU-funded NAIAD project through our colleagues in France and we are exploring future impact studies internationally. In our recently submitted article, a basic numerical model of Wetropolis is used to determine the relevant time and length scales prior to its construction as a physical model. We are developing the hydrodynamic modelling further, both mathematically and numerically, in order to conduct idealised experiments in flood control and mitigation.

Image  ‘FEV concept’

Presentations

  • Bokhove, O., Kelmanson, M. A., Kent, T., Piton, G., & Tacnet, J. M.: Using flood-excess volume to assess and communicate flood-mitigation schemes. EGU general assembly, Vienna, April 2019 (oral). Available online.
  • Bokhove, O., Kent, T., de Poot, H., & Zweers, W.: Wetropolis: models for education and water-management of floods and droughts. EGU general assembly, Vienna, April 2019 (poster). Available online.
  • Kent, T., Cantarello, L., Inverarity, G., Tobias, S.M., Bokhove, O. (2019): Idealized forecast-assimilation experiments and their relevance for convective-scale Numerical Weather Prediction. EGU general assembly, Vienna, April 2019 (oral). Available online.
  • Bokhove, O., Kelmanson, M. A., Kent, T., Piton, G., & Tacnet, J. M.: Public empowerment in flood mitigation, Flood & Coast conference, Telford, June 2019 (oral).
  • Bokhove participated in the ‘Landscape decisions’ program at the Isaac Newton Institute, Cambridge (July/August 2019). Web: https://www.newton.ac.uk/event/ebc
1st NCEO-GSSTI Data Assimilation and Earth Observation Training Course

1st NCEO-GSSTI Data Assimilation and Earth Observation Training Course

by Javier Amezcua

25 November 2019

I spent the last week in Accra, the capital of Ghana. It was incredibly hot and stuffy for this time of the year (minimum 26C, maximum 31C), which is natural if we consider this city is only 5N of the Equator. The food was delicious (I stuffed myself with joloff rice and fried fish) and I enjoyed sunsets when colonies of bats flew over the city.

In this trip I was accompanied by Ewan Pinnington and Tristan Quaife from University of Reading, and Jose Gomez-Dans from University College London. We were in a mission for the UK National Centre of Earth Observation (NCEO), to which the four of us belong. Our mission was to deliver a training course in data assimilation and Earth observation for the young Ghana Space Science and Technology Institute (GSSTI). This institute is located in the northern outskirts of Accra in the campus of the School of Nuclear and Allied Sciences. The participants of the course included people from GSSTI, the Ghana Statistical Institute, the Ghana Meteorological service, and a member of the United Nations Food and Agriculture Organisation (FAO).

This course is part of the continuous collaboration between scientists of the UK and Ghana under the Official Development Assistance (ODA). This program exhorts developed countries to dedicate a percentage of their gross domestic product (GDP) as aid to help foster prosperity in developing countries. This scheme was started by the Organisation for Economic Co-operation and Development (OECD). A country can participate directly with monetary aid, but also through knowledge and expertise. Our training course belongs to the latter category.

In the course I went through the fundamental aspects of data assimilation: definining the estimation and forecasting problem, revising some basic concepts of probability and statistics, and emphasizing the role of Bayes’ Theorem as a central aspect of data assimilation. I then explained some of the basic families of data assimilation methods: variational and Kalman-based. We did some computer experiments with a toy model in order to illustrate some ideas.

My colleague Jose Gomez-Dans then presented something more specific to fulfill the needs of our audience. In particular, people were quite interested in using satellite observations to infer the conditions of crops in the north of Ghana, and then use land-surface models to predict the yield at the end of the season. These models contain information about the biology of the crops, human activities, and they are forced by meteorological products. He helped the participants run some experiments remotely from some computers from UCL with observations from the Sentinel Mission of the European Space Agency (ESA).

We had a great week and it the course was very well received by the participants. It was rewarding to see science transcending the ever tighter borders, institutions opening doors instead of closing them, and people collaborating instead of fighting. We hope to continue our collaboration with GSSTI, and we are planning on coming back in 2021.

Investigating the uncertainty of weather radar data

Investigating the uncertainty of weather radar data

This blog describes the work of Masters student Vasiliki Kouroupaki, carried out in collaboration with the UK Met Office.

UK weather radar network

In numerical weather prediction, nowcast, hindcast and forecast models can be improved through data assimilation. Data assimilation is the technique which combines observations with output from a previous short-range forecast (background) to produce an optimal estimate of the state of the atmosphere (analysis).

Radar reflectivity observations are assimilated by the Met Office in order to provide up-to-date information about rainfall in the initial conditions for UK weather forecasts. In assimilation, observations are assigned weights according to their error statistics. Depending on the kind of observation, there are different factors or processes which can result in errors. In order to have an optimal analysis these errors must be correctly specified. However, due to the fact that the true errors are not known their statistics need to be estimated. In this work, the uncertainties of radar reflectivity observations assimilated into the Met Office UKV model are examined using a diagnostic technique. Data come from the operational UKV model with hourly cycling 4D-VAR or from trial experiments four times per day. The diagnostic is based on combinations of observation-minus- background, observation-minus-analysis and background-minus-analysis differences. The results show that observation error variances are higher for Winter (1 Dec 2017-18 Jan 2018) than for Summer (16 Jul-16 Aug 2018) and that they increase for higher reflectivity values. Further investigations classified the data by beam elevation and by radar ID. These showed that for values of beam elevation between 0.5- 1.0 and 3.0-4.0 degrees the error variance had greater values. Also, error statistics for different radars were positively correlated with the mean reflectivity observed by each radar.

Further investigation of observation error statistics in the assimilation could improve the initial conditions and thereby operational forecasts for convective rainfall events.

Summer school on Data Assimilation and its applications in oceanography, hydrology, risk&safety and reservoir engineering, 2019

Summer school on Data Assimilation and its applications in oceanography, hydrology, risk&safety and reservoir engineering, 2019

by Haonan Ren, PhD student in Atmosphere, Oceans & Climate, University of Reading
August 14, 2019

From 22nd July to 2nd August, the Summer School on Data Assimilation and its applications in oceanography, hydrology, risk&safety and reservoir engineering was held in the Faculty of Mechanics, Polytechnic University of Timisoara, Romania. This two-week summer school has been organized every two years since 2009, and targets primarily students and researchers at an early stage of their career with/without previous experience in data assimilation. In the 6th DA summer school, there were 35 participants from universities, research institutes and industry from all over the world.

The goal of this summer school is to gather the experts in the field of data assimilation from different disciplines (statistics, pure mathematics, engineering, etc), and use their knowledge to educate so that the participants can get some basic knowledge of data assimilation and its applications and have a taste of the advantages of using the data assimilation in different fields. Furthermore, the participants can also work hands-on with academic and commercial dedicated software, and have extensive discussions and exchange ideas with the instructors and other participants. The lectures in the first week focused on the theoretical framework of data assimilation. The lectures started with some basic concepts and derivations of Kalman Filter (KF), including the motivation of using data assimilation in different fields. Then, a Monte-Carlo formulation (ensemble) of KF was introduced, Ensemble Kalman Filter (EnKF), including the necessary processes needed when using EnKF in practicals, such as localization and inflation.

The lectures for rest of the first week demonstrated another method of data assimilation, Particle Filter (PF), and showed the general ideas of data assimilation for chaotic systems and dynamical system. Each day, after a morning of intensive lectures, there was a two-hour practice in the afternoon, in which the participants were given some exercises based on Baye’s Theorem and got the opportunities of running data assimilation schemes on simple models using different programming frameworks. The practicals were strongly connected to the lectures, so that the students could have a better understanding of data assimilation.

The summer school arranged lunches at a local restaurant which was walking-distance from the university, and the organizers of the summer school booked local restaurants near the city centre of Timisoara at the end of the day, so that both the instructors and students could get some relaxation after an exhausting day, and enjoy the local cuisine and cold beers. During the weekend of the first week, the summer school provided a trip around the border of Romania, which involved hiking and sightseeing of the natural landscape of Romania.

 

 

 

 

 

 

 

 

 

 

 

 

 

After a relaxing weekend, the following week concentrated on the applications of data assimilation in different areas. The instructors started with some fundamental knowledge about computer science in different programming languages, followed by demonstrating the numerical schemes for numerical models. Then, the lectures specifically looked at the applications of data assimilation on the ocean and climate models. During the lectures, the instructors also gave some basic knowledge about oceanography and climate, which gave the students a better insight into the models for a real world application. In the second week, there were several lectures discussing the application of the Ensemble Kalman Smoother (EnKS) and other methods in reservoir engineering (oil and gas), and decision-making problems. And at the end of the final week, the lectures were introducing the field of big data, and the geomechanical applications of data assimilation scheme.

This summer school offered a fulfilling experience about data assimilation, both in theoretical framework and practical applications, to all the participants. And for both instructors and students, the summer school also provided an opportunity to discuss their work and change opinions and experience.

I would like to thank the EPSRC DARE project and Prof. Sarah Dance for the funding that enabled me to attend this summer school.

Data assimilation training at the University of Reading

Data assimilation training at the University of Reading

by Amos Lawless

In March 2019 the Data Assimilation Research Centre at the University of Reading organised a 4-day training course in data assimilation, in collaboration with the National Centre for Earth Observation, ECMWF and the DARE project. The course was attended by 24 early-career researchers from 10 different countries, including scientists from universities, research institutes and industry.

The aim of the course was to give students a solid grounding in the theory of data assimilation methods, as well as the opportunity to apply data assimilation methods to a range of numerical models. The first day of the course saw a general introduction to data assimilation, followed by a more in-depth look at variational methods, both from a theoretical and practical point of view. A computer practical session in the afternoon gave students the opportunity to deepen their understanding by running a variational scheme on a simple numerical model. The day ended with an ice-breaker event, allowing attendees to discuss their particular research projects and their interest in data assimilation over a drink and some nibbles.

The remainder of the course looked at the theory and practice of other data assimilation methods, each supported by computer practical sessions, including the ensemble Kalman filter on day 2, hybrid methods on day 3 and the particle filter on the final day. In between students were treated to two lectures on practical applications: PhD student Jemima Tabeart spoke of her work looking at observation error correlations in the Met Office 1DVar data assimilation system, while research fellow Polly Smith spoke about coupled atmosphere-ocean data assimilation. At the end of day 3 a group meal was organised at the Zerodegrees microbrewery restaurant in the centre of Reading, giving further opportunity for informal discussions of the course material and how students could use the ideas in their own projects. At the end of the final day, after attendees were presented with their course certificates, the staff were also presented with a gift – A 900-piece Lego wind turbine from the Danish attendees. So, if you don’t hear from us for a while, you will know why!

All lecture notes from the course and material for the computer practicals are available to download from the course web site

https://research.reading.ac.uk/met-darc/ecmwf2019/

 

 

 

ISDA2019 in Japan

ISDA2019 in Japan

by Dr Natalie Douglas, University of Surrey and Dr Alison Fowler, University of Reading and NCEO

ISDA2019, the 7th International Symposium for Data Assimilation 2019, was hosted in Kobe, Japan this year from the 21st to the 24th January at the RIKEN Center for Computational Science – home of the K-computer. Attended by over 100 research scientists, the conference boasted a guest list of inspiring speakers and poster presenters from all over the globe. Topics of current relevance invoking enthusiastic discussion included Big Data Assimilation, Uncertainty Quantification, Satellite and Coupled DA, Multi-Scale Processes and DA in Broader Applications to name a few.

“I thoroughly recommend attending ISDA to anyone working in Data Assimilation. This was my first conference abroad, it was hugely informative and extremely well organised. Not only that, I had enormous amounts of fun getting to know and even making good friends with a lot of the key players in my field.” – Dr Natalie Douglas from the University of Surrey, UK.

“The ISDA provided a fascinating overview of the latest developments in Data Assimilation from around the world. It included a diverse range of applications from supernova astrophysics to my more familiar area of meteorology. I found the chance to spend a week with other scientists hugely beneficial to my work. After the symposium I enjoyed an extended visit to RIKEN to continue discussions on the efficient use of high-volume observations in their home-developed rapid-update-forecasting system. The aim of this state-of-the-art system is to provide advanced warnings of the most extreme rainfall events that can evolve in the matter of minutes. Each year in Japan, such events result in a multitude of deaths and wider devastation, and so such a system is sorely needed. Bringing this knowledge back to the UK may prove greatly beneficial as we prepare for the effects of a changing climate.”  – Dr Alison Fowler from the University of Reading, UK and NCEO.

Flood Inundation Mapping with Data Assimilation or Summary of Zhiqi Hu MSc thesis

Flood Inundation Mapping with Data Assimilation or Summary of Zhiqi Hu MSc thesis

Due to climate change flooding is predicted to increase in frequency and intensity across the globe and it is imperative we can produce accurate and timely flood forecasts for decision-makers before and during floods.

Zhiqi Hu, an MSc in Atmospheric Ocean & Climate student at University of Reading, worked with us and JBA Consulting during her masters project investigating if a probabilistic ensemble weighting method can improve  Flood Foresight ensemble flood map forecasts using satellite observations during the flood event in India, Brahmaputra river basin in August 2017. Her work is summarised in this poster.