The Need for Harmony

by Tim Trent

October’s blog (Is gracious wobbling the new dancing?) ends with the important question, “Is the error structure of our climate data records adequately characterised, or not?”  This question is central to the FIDUCEO project, and is key to understanding changes in the climate system from satellite observations. This month we take a look at 2 key results from the recent GEWEX Water Vapor Assessment (G-VAP)[1] to demonstrate why this question is so important. The G-VAP working group was initiated in 2011 and is an international collaborative effort involving scientists from all over Europe and the USA. The remit of G-VAP has been to perform the first comprehensive assessment of water vapour climate data sets, assessing trends from a number of long-term records.

So why is water vapour important?

Water vapour is an important greenhouse gas within the atmosphere as it influences (directly and indirectly) the radiative balance of the Earth as well as surface fluxes and soil moisture. However, it is sufficiently abundant and short-lived that it is essentially considered to be under natural control [2]. Water vapour also has a predominant capacity for positive feedback that acts as the largest amplification mechanism for anthropogenic climate change compared to radiative forcing from greenhouse gases [3]. The water vapour signal is dominated by the troposphere, where concentrations vary by 4 orders of magnitude between the tropopause (8-12 km) and the surface. The role of tropospheric water vapour in atmospheric processes extends over a wide range of spatial and temporal scales, from the global climate to micrometeorology [4].

What are the challenges in building a climate time series?

Total column water vapour (TCWV) measurements from satellites form our longest global records of absolute concentrations (of water vapour). With 80% of the signal contained in the lowest 2 km of the column, TCWV is sensitive to changes in the lower troposphere. From the Clausius–Clapeyron relation we expect a 6% to 7.5% per Kelvin per decade change in water vapour. Therefore, this inherently means to observe any water vapour signal at climate scales requires the combination multiple satellite records. And it is at this point where the harmonisation of records becomes important [5]. Operational sensors have periods where there is an overlap (in operation) of the current and replacement sensor. Differences in calibration and performance between the two sensors during these periods introduce step functions into a time-series. Considering some records such as reanalysis that use a range of different sensor types, this can very quickly become very complicated issue especially if we want to look at long term trends.

Results from G-VAP

In the G-VAP project, the homogeneity of TCWV records that were longer than 20 years was examined. This exercise used gridded monthly mean data from satellite and reanalysis products over global ice-free oceans and subjected them to a penalized maximal F test (PMFT) [6]. This test is used for detecting shifts that are not accompanied by any sudden change in the linear trend of the difference in the anomaly time series between a record and the reference. The Hamburg Ocean Atmosphere Parameters and Fluxes from Satellite Data (HOAPS) SSM/I [7] record was used for this purpose as it based on homogenised observations [8] from the Special Sensor Microwave Imager (SSM/I).

Figure 1: Identified break points in 5 TCWV anomaly time series over global ice free oceans, adapted from Figure 2 Schröder et al (2016). Each time series has had a penalised maximal F test applied to anomaly differences relative to HOAPS SSM/I record. Records include (top left) ERA-Interim (ERAINT), (top centre) REMSS, (top left) NCEP (CFSR), (bottom left) MERRA and (bottom centre) NVAP-M. The PMF fit line is shown in red, the anomaly differences in black and the dates where breaks occur are shown in the top right legend in each plot. Note that y axis scales differ between plots.

The global results of this analysis are shown in Figure 1. What is immediately clear is that each record has break points and they vary from data set to data set. Investigation of these break points revealed that they appear when there is a change in the satellite instrument observation system(s) being used [9]. Further breaks were found when looking at regional compared to the global results. One caveat of these results arises when switching from ocean to land based comparisons is that the reference used changes. Therefore, understanding the uncertainty of the whole (comparison) system is needed to fully characterise performance. However, what we can truly say is that the current understanding of the uncertainty in these measurements is not good enough to produce consistent climate data records.

Figure 2: Adapted from Figure 4-36 of the G-VAP report [1] showing trend estimates of TCWV in % per Kelvin per decade for ten different data records. TCWV values were averaged over the global ice-free oceans between ±60°N/S. The uncertainty of the trend regression are shown as vertical bars. The dashed black lines show the theoretical expectation at 300 K (bottom) and 275 K (top), while the green dashed line marks the actual expectation based on the mean change in SST.

This assertion is further corroborated by G-VAP when the trends in these records were examined. For global ice-free oceans, trends were estimated by applying a model that accounts for El Nino strength and the asymmetric fitting of the annual cycle [10]. These results showed significant differences between the records (Figure 2). From the 10 records that were considered only 2 are in line within expected trend estimates (6-7.5 %/K/decade), while the others vary from 2.6 %/K/decade (JRA55) to 24.9 %/K/decade (CFSR). In addition to global estimates, regional trends were also analysed. From both of these scenarios, G-VAP concluded that:

  1. At the global ice-free ocean scale, trend estimates among the 10 records exhibit large differences and are frequently significantly different.
  2. Except for the 2 SSM/I records (HOAPS and REMSS), all data records exhibit regression values outside the theoretically expected range. This is an indication of issues in long-term stability.
  3. The differences in the trend estimates on regional and global ice-free ocean scales were found to be caused by breakpoints or series of breakpoints. Generally these breakpoints coincide temporally with changes in the observing system.
  4. The time, sign, and step size of breakpoints are typically a function of region and data record. In particular, the breakpoint characteristics are different between time series from the regional and the global ice-free ocean scale. The imprint of changes in the observing systems is a function of region.
  5. The majority of these breakpoints are not evident against comparisons to radiosonde soundings. One reason for this is due to areas with distinct differences in trend estimates are not covered with stations.

The importance of Harmony

These results clearly demonstrate the impact heterogeneity in records can have, and for an essential climate variable like water vapour can make them inadequate for climate analysis if not properly characterised. The later stages of the FIDUCEO project will generate new climate data records from fully characterised FCDR’s. Although TCWV is not currently among these, the concept of how the long term stability can impact on observed trends is still valid. Testing the performance of CDRs generated from the harmonised FCDRs in a comparative manner will demonstrate the potential of “FIDUCEO” CDRs to assess climate processes. Lessons learned from this process will provide a framework in which further CDRs can be produced in the future.


[1] M. Schröder, M. Lockhoff, L. Shi, T. August, R. Bennartz, E. Borbas, H. Brogniez, X. Calbet, S. Crewell, S. Eikenberg, F. Fell, J. Forsythe, A. Gambacorta, K. Graw, S.-P. Ho, H. Höschen, J. Kinzel, E. R. Kursinski, A. Reale, J. Roman, N. Scott, S. Steinke, B. Sun, T. Trent, A. Walther, U. Willen, Q. Yang, 2017: GEWEX water vapor assessment (G-VAP). WCRP Report 16/2017; World Climate Research Programme (WCRP): Geneva, Switzerland; 216 pp.

[2] Sherwood, S.C., Roca, R., Weckwerth, T.M. and Andronova, N.G., 2010. Tropospheric water vapor, convection, and climate. Reviews of Geophysics, 48(2).

[3] Chung, E.-S., B. Soden, B. Sohn, and L. Shi (2014), Upper-tropospheric moistening in response to anthropogenic warming, Proceedings of the National Academy of Sciences, 111(32), 11,636-11,641.

[4] Bevis, M., Businger, S., Herring, T.A., Rocken, C., Anthes, R.A. and Ware, R.H., 1992. GPS meteorology: Remote sensing of atmospheric water vapor using the Global Positioning System. Journal of Geophysical Research: Atmospheres, 97(D14), pp.15787-15801.

[5] ‘Harmonisation and Recalibration’, FIDUCEO blog by Emma Woolliams, 22-06-2016,

[6] Wang, X.L., 2008. Penalized maximal F test for detecting undocumented mean shift without trend change. Journal of Atmospheric and Oceanic Technology, 25(3), pp.368-384.

[7] Andersson, A., Fennig, K., Klepp, C., Bakan, S., Graßl, H. and Schulz, J., 2010. The Hamburg Ocean Atmosphere Parameters and Fluxes from Satellite Data–HOAPS-3, Earth Syst. Sci. Data, 2, 215–234.

[8] Fennig, K., Andersson, A., Bakan, S., Klepp, C. and Schröder, M., 2012. Hamburg Ocean Atmosphere Parameters and Fluxes from Satellite Data-HOAPS 3.2-Monthly Means/6-Hourly Composites.

[9] Schröder, M., Lockhoff, M., Forsythe, J.M., Cronk, H.Q., Vonder Haar, T.H. and Bennartz, R., 2016. The GEWEX Water Vapor Assessment: Results from Intercomparison, Trend, and Homogeneity Analysis of Total Column Water Vapor. Journal of Applied Meteorology and Climatology, 55(7), pp.1633-1649.

[10] Mieruch, S., Schröder, M., Noël, S. and Schulz, J., 2014. Comparison of decadal global water vapor changes derived from independent satellite time series. Journal of Geophysical Research: Atmospheres, 119(22).