Harmonisation

By Ralf Quast and Sam Hunt

Theoretical basis

Harmonisation designates the conceptual framework behind FIDUCEO Fundamental Climate Data Records. A harmonised satellite data series is one where the calibration of all sensors in a series of sensors have been made consistent with a reference dataset which can be traced back to known reference sources, in an ideal case back to SI. Each sensor is calibrated to the reference in a way that maintains the characteristics of that individual sensor such that the calibrated radiances represent the unique nature of each sensor. This means that two sensors which have been harmonised may see different signals when looking at the same location at the same time where the difference is related to known differences in the responses of each sensor such as differences in the sensor spectral response functions. Harmonisation can be achieved to within an uncertainty that should be estimated, and the harmonisation uncertainty contributes to the component of uncertainty that is common across the whole data record of each sensor.

We define recalibration as obtaining new calibration coefficients, and possibly even new calibration models. Both may be done by comparing the output of one satellite to radiometrically more accurate sensors using appropriate match-ups like simultaneous nadir overpasses. Recalibration goes beyond bias correction, which has the same aim, but performs differently. Recalibration adjusts the calibration coefficients, leading to new measured values, whereas in bias correction, an offset or factor is applied to the existing measured values. In particular harmonisation does not aim to ‘correct for’ spectral response function (SRF) differences by translating the measured value of the test sensor as though they were taken by the reference sensor (‘homogenisation’). Instead harmonisation aims to reconcile the calibration of all sensors given their estimated SRF differences.

Namely, let \textcolor{orange}{L^i} and \textcolor{red}{L^j} denote the calibrated radiance measured by two different sensors \textcolor{orange}{ i} and \textcolor{red}{j} which look at the same Earth target at the same time and let \textcolor{purple}{ K^{ij}} denote the radiance difference we expect because the two sensors have different spectral response functions. Then harmonisation imposes this expected difference

\textcolor{orange}{L^i} – \textcolor{red}{L^j} = \textcolor{purple}{PK^{ij}} \tag{Eq. 1}

whereas homogenisation ignores it

\tag{Eq. 2} \textcolor{orange}{L^i} – \textcolor{red}{L^j} = 0

Mathematically, harmonisation is equivalent to minimising the difference between the left hand side and the right hand side of Eq. (1). The conceptual differences between harmonisation and other methods often practiced in Earth Observation are summarised in Figure 1.

Figure 1: Harmonisation compared to other methods often practiced.

Why harmonisation?

Harmonisation considers long-standing historic sensor series. For any such sensor, the behaviour in orbit can be very different from its behaviour during pre-launch testing and more scientific value can be derived from considering the sensor series as a whole, for both Fundamental Climate Data Records and Climate Data Records derived thereof.

For example, let us consider the Advanced Very High-Resolution Radiometer (AVHRR) onboard the NOAA and MetOp series of satellites. The first satellite in this series was launched in the late 1970s while the last was launched in 2012, leading to about 40 years of Earth radiance data (Figure 2). The similar Advanced Along Track Scanning Radiometer (AATSR) onboard Envisat was considered the state of the art for many years and provides a community-accepted reference.

Figure 2: Series of NOAA and MetOp satellites equipped with the AVHRR. Envisat is shown as the reference.

Looking at the calibration residuals (or K-residual)

\tag{Eq. 3} \textcolor{green}{r^{ij}} = \textcolor{orange}{L^i} – \textcolor{red}{L^j} – \textcolor{purple}{K^{ij}}

for match-ups (e.g. simultaneous nadir overpasses) of operationally calibrated Earth radiance from many different sensor pairs (top panel of Figure 3) reveals severe metrological problems in form of systematic jumps and trends over time. These problems, however, virtually disappear after harmonisation (bottom panel of Figure 3).

Figure 3: Random subset of calibration residuals for the 11-micron channel of AVHRR before(top panel)  and after harmonisation.

How does FIDUCEO help?

An explicit methodology to conduct such a harmonised calibration of a series of sensors has not existed before FIDUCEO and was explored and developed by following the errors-in-variables (EIV) idea, which takes into account the uncertainty and error correlation in all sensor telemetry measurements and yields optimised calibration coefficients and an associated calibration error covariance matrix for all sensors (e.g. Newey 2001).

Figure 4 broadly illustrates the methodology which was developed by the FIDUCEO project to conduct harmonisation within four steps. Step one is to collect dual-sensor match-ups where two sensors observe approximately the same Earth target at approximately the same time (e.g. simultaneous nadir overpasses). Ideally, these match-up data include all possible reference-to-sensor and sensor-to-sensor pairs. Step two is an assessment of the error covariance of all measurements included with the match-up data and an assessment of the expected match-up radiance differences and their associated uncertainties. Step three consists in a joint optimisation of sensor calibration models. The last step is the evaluation of the calibration coefficient error covariance matrix. The complete methodology is described in detail by Giering et al. (2019) and explained at a higher level in the following sections.

Figure 4: Methodology to conduct harmonisation

Newey, W.K. Flexible simulated moment estimation of nonlinear errors-in-variables models. Rev. Econ. Stat. 2001, 83, 616–627. [CrossRef] Giering, R.; Quast, R.; Mittaz, J.P.D.; Hunt, S.E.; Harris, P.M.; Woolliams, E.R.; Merchant, C.J. A Novel Framework to Harmonise Satellite Data Series for Climate Applications. Remote Sens. 2019, 11, 1002. [CrossRef] Harris, P.M.; Hunt, S.E.; Quast, R.; Giering, R.; Mittaz, J.P.D.; Woolliams, E.R.; Dilo, A.; Cox, M. Solving large structured non-linear least-squares problems with an application in Earth observation. In preparation.