1. Determining the Measurement Function

Link to the CEOS Cal/Val portal where this tutorial http://www.fiduceo.eu/tutorial/introducing-measurement-function will eventually live

What is the Measurement Function?

All FCDRs are calculated from a measurement function. For all FIDUCEO FCDRs the measurement function is an equation, but in principle it could also be an algorithm described in code rather than as an equation. The equation calculates the FCDR measurand (e.g. radiance, reflectance or brightness temperature) from raw measurement counts and calibration parameters (e.g. gains, offsets and non-linear parameters). Some of these calibration parameters may be set from pre-launch (or design) considerations, others will be established in orbit (e.g. from an on-board or vicarious calibration approach) and some will be established retrospectively through harmonisation processes.

How do you derive the Measurement Function?

Level 1 (L1) radiances are typically calculated using an equation similar to:

$L_{c,l,e} = a_{0}+a_{1}C^{E}_{c,l,e}+a_{2}(C^{E}_{c,l,e})^{2}+0$ (Eq.1)

where;

$L_{c,l,e}$ is the calculated Earth radiance for channel c of the pixel at image co-ordinate (l,e)
$C^{E}_{c,l,e}$ is the Earth count for the pixel recorded by the sensor.
$a_{0},a_{1},a_{2}$ are calibration parameters with $a_{T}=[a_{0}, a_{1}, a_{2}]$

Uncertainty in the calculated radiance arises in part from uncertainties in the values of the quantities on the right hand side of the equation. In general, there are also other effects that are expected to have zero mean and which contribute uncertainty in the calculated radiance; the “+ 0” is included as a reminder of this. (Note that Eq. 1 is not representative of the calculation of radiance spectra from interferometers.)

In general, the calibration parameters are determined by in-flight calibration data plus other information. Sensors are generally reasonably linear, so a₂ is often small;
a₁ is the ‘gain’ of the sensor. In-flight calibration systems typically have two reference points, such as a dark and bright target, to estimate changes in gain over time in flight. With two references, characterisation of non-linearity needs external information, such as pre-flight characterisation or in-flight characterisation against another sensor.

Changes in the gain and offset are to be expected as the sensor’s space environment changes and the sensor degrades. An important source of change in the space environment is any precession of the orbit plane relative to the Sun (changing local solar time) over the mission lifetime. Material properties of components of the in-flight calibration system may degrade in time. The uncertainties associated with measured radiances will therefore evolve, and the stability of measurement may not be calculable from the satellite data alone.

Typically, 3 to 10 years after launch, the sensor will fail or the platform will be decommissioned. Multi-decadal datasets are built from sensors on a series of missions. Ideally, the sensors in a series would have identical spectral response and missions would overlap by a year or more. In practice, nominally equivalent channels have significant differences in SRFs and the overlaps between missions are not fully controllable.

Why do we need to do this?

Metrologists may be surprised that uncertainty estimates are not routinely included in L1 products under current practices in EO. A metrological approach to L1 products would include adopting the principle that every measured value in an L1 product should have associated context-specific uncertainty information (provided per datum if necessary). Although this principle (for Level 1 and higher products) and a set of associated guidance was endorsed by CEOS (Committee on Earth Observation Satellites) in 2010 as part of the Quality Assurance Framework for Earth Observation (QA4EO) , this is still in the process of adoption at all space agencies.

Uncertainty analysis in the GUM begins with modelling the measurement, i.e., linking the measurand to the input quantities from which it is derived. A generic measurement model for a L1 radiance would be:

$\tag{Eq. 2} Y=f(X_{1}, X_{2},…,A)+Δ$

where:

$X_{1}, X_{2},…$

are input quantities;

$A$

is the vector of calibration parameters, which are also input quantities but are usefully distinguished; and

$Δ$

is an input quantity introduced to represent any inadequacy of the function f to represent all phenomena that affect the measurand.

The equations, such as Eq 1 used to populate L1 products, evaluate the measurand (radiance) using estimates of the input quantities. In the GUM, the convention is for estimates to be represented with the lower-case characters corresponding to the quantities written in upper case. Eq. 1 is then seen as a particular case of the expression by which the measurand is estimated:

$\tag{Eq. 3} y=f(x_{1}, x_{2},…,a)+δ$

where the input estimates include the recorded sensor counts, etc. This clarifies the meaning of the ‘+0’ term previously introduced: 0 is our best estimate of

$δ$

which is the expectation of

$Δ$

(assuming we are using the best measurement model we can formulate). The uncertainty in the measured value is derived from the evaluation of the uncertainty in each input estimate (or, strictly, from the distribution of possible values of $X_{i}$ given $x_{i}$ ).

The uncertainty in all input estimates, including calibration parameters and δ is relevant. Evaluation of the uncertainty in y means propagation through the measurement model of these uncertainties (or, strictly, distributions).

How do you improve the measurement function?

In order to assess and improve the measurement function, it is important to iterate the function and to incorporate harmonisation information into the process.

In general, a generic measurement model for a L1 radiance would be

$\tag{Eq. 4} Y=f(X_{1}, X_{2},…,A)+Δ$

where:

$X_{1}, X_{2},…$

are input quantities;

$A$

is the vector of calibration parameters and

$Δ$

is an input quantity introduced to represent any inadequacy of the function f to represent all phenomena that affect the measurand.

Here, the $X_{i}$ (or the $CE_{c}(l,e)$ ) are the instantaneous input parameters – usually the Earth count signal and possibly other signals from the sensor. The calibration parameters A convert these Earth counts into the measurand – Earth radiance. They include the gain, any corrections for nonlinearity, or other corrections from the instrument characterisation (e.g. antenna pattern corrections in the microwave). These may be determined:

Prelaunch – based on the laboratory characterisation and calibration of the instrument
Postlaunch – from onboard calibration using either onboard sources or reflectors or vicarious methods
In FCDR production through harmonisation

In practice a combination of these will be used. Prelaunch methods are established before the mission, postlaunch during operation and harmonisation methods use additional information from match ups with other sensors to “back correct” historical data based on new information.

Harmonisation recalibrates a sensor based on overlaps with other sensors to get a stabilised long-term record from a series of sensors. This involves redetermining new values for some of the calibration parameters. The choice of which calibration parameters to determine through harmonisation, and which to determine based on pre or post launch methods, and indeed, the choice of the form of the equation (and therefore which calibration coefficients are within it, e.g. whether to do nonlinearity with only a second or also a fourth order term) may be made iteratively – e.g. following the full process of FCDR uncertainty analysis, determining effects tables and harmonisation, it may be appropriate to repeat the process with a different form of measurement function, or with different coefficients determined from harmonisation. N.B. the process of metrological uncertainty analysis may highlight problems with the measurement function or the approach taken and lead to corrections being applied.

The FIDUCEO harmonisation approach requires consideration of the error correlation structure of the observational data. Furthermore, the different terms, having been obtained from the same observational data, are usually correlated with one another. The degree of correlation is determined during the harmonisation process. If it is extremely high it may not be meaningful to determine the two highly correlated harmonisation parameters separately and in that case the measurement equation may need reconsideration.

The FCDR measurement function typically contains calibration parameters that have been determined from a calibration process performed during the FCDR development. These calibration parameters generally represent physical attributes of the instrument (e.g. its nonlinearity, the inflight degradation of mirrors, or a stray light correction) where better calibration information is available for the post-launch situation through comparisons with a reference than is available from applying the pre-flight calibration. Such calibration parameters may be used to account for an observed systematic effect that is inherent to the instrument, for example calibration drift over time, or for empirically demonstrated systematic effects (e.g. instrument temperature sensitivity) even where the exact physical cause cannot be determined.

Example

In the FIDUCEO project, the MVIRI sensor drifts and biases are corrected by calibrating the instrument against a small number of stable ground sites whose top-of-atmosphere (TOA) radiance had been determined using a radiative transfer model. For the other FIDUCEO sensors, the recalibration was obtained using harmonisation: where match-ups with other sensors in the series and where a reference sensor is used to determine physical-origin calibration coefficients (the harmonisation coefficients) in the measurement function. In all these cases a set of harmonisation/calibration parameters was determined by fitting a model to observational data.

Useful documents

Merchant C. & Woolliams, E., 2017, Mathematical notation for FIDUCEO publications

http://fiduceo.pbworks.com/w/file/121751748/Notation%20FIDUCEO%20v1a.pdf