Data Correlation Structures
Means that the error in a measured value is considered to be a stochastic independent draw from an underlying probability distribution; “random” implies in this context both “unpredictable” and “uncorrelated across measurements”; random errors therefore tend to “average out” across many measured values; random effects may be operating at the same time as other types of effect, in which case only a component of the total error is random; an example of a random effect (an effect giving rise to random errors) is electronic noise in an amplifier circuit.
Means that the error in a measured value is determined by dependence on some factors; systematic error could in principle be corrected for if the dependencies were understood and the factors were known; where the factors vary negligibly across many measurements, the errors from the systematic effect are the same; “systematic” implies “predictable” (in principle, not in practice) and “correlated across measurements”; systematic errors therefore “average out” slowly or not at all across many measured values; systematic effects may be operating at the same time as other types of effect, in which case only a component of the total error is systematic; an example of a systematic effect is a mis-characterised calibration target.
means that across many observations there is a deterministic pattern of errors whose amplitude is stochastically drawn from an underlying probability distribution; “structured random” therefore implies “unpredictable” and “correlated across measurements”; the degree of “averaging out” across many measured values depends on the structure of the effect across those measured values; structured random effects may be operating at the same time as other types of effect, in which case only a component of the total error is structured random; an example of a structured random effect is the impact of a random error in the measurement of signal while viewing a calibration target, which causes unpredictable but inter-related errors in all measured values which use that calibration cycle.
Locally systematic or locally correlated
a particular case of structured random, where measured values obtained together (having small separations in time and space) have highly correlated, similar magnitude errors, whereas errors in measurements separated by longer space-time scales are independent and uncorrelated.
A harmonised satellite series is one where all the calibrations of the sensors have been done consistently relative to reference datasets which can be traced back to known reference sources, in an ideal case back to SI. Each sensor is calibrated to the reference in a way that maintains the characteristics of that individual sensor such that the calibration radiances represent the unique nature of each sensor. This means that two sensors which have been harmonised may see different signals when looking at the same location at the same time the difference being related to known differences in the responses of each sensor such as differences in the sensors spectral response functions etc.
Unlike harmonisation, homogenisation is where all satellites are forced to look the same such that when looking at the same location at the same time they would (in theory) give the same signal. In reality the signals from different sensors would be different and homogenisation is adding in corrective terms to each satellite to make them look the same. It is likely that these corrective terms will not be 100% effective and that the process of homogenisation will add in scene dependent errors to the uncertainty budget which may be difficult to assess.
Intercalibration is the process of cross comparing one satellite with another dataset used as a reference. Often the reference dataset is another satellite whose calibration is better characterised and/or updated relative to the satellite of interest and so can be used to recalibrate and/or provide information on problems/biases in the satellite of interest.
A recalibrated dataset is one where the calibration has been updated relative to the operational calibration used in the original satellite data. The operational calibration is normally derived from pre-launch measurements and there are many instances where the pre-launch data is insufficient to calibrate the sensor in orbit either due to changes in the satellite response while in orbit or due to problems with the pre-launch data itself or both.
Radiometric normalisation is a process which attempts to remove some of the variance seen in EO data to create values that are independent of view angle or atmospheric state or observing time etc. to give a uniform measure of a given variable across an image. It can be considered as a method to give what would have been observed by the same instrument under viewing identical conditions.
Traceability And Uncertainty
Traceability is defined by the Committee of Earth Observation Satellites (CEOS) as: Property of a measurement result relating the result to a stated metrological reference through an unbroken chain of calibrations of a measuring system or comparisons, each contributing to the stated measurement uncertainty. Traceability involves both an unbroken chain to that reference – a clear link of “A was calibrated against B, which was calibrated against C and so on to the reference” and the documentary evidence that each step was performed in a reliable way, with clear uncertainty analysis in the form of an uncertainty budget for each step. Ideally this documentation is reviewed through peer review or formal audit. Note that there are other common uses of the term “traceability” including that it is possible to “trace” the origin of all the input data sets and that there are appropriate algorithmic documents (e.g. ATBDs) and that software is formally checked. These are all important aspects of a quality system. Metrological traceability is a further step beyond this.
SI-Traceability is traceability where the “stated metrological reference” is formally calibrated within the International System of Units (SI) through a National Metrology Institute that participates in the Mutual Recognition Arrangement and whose measurement for this parameter is audited through formal international comparison and peer review.
The GUM defines uncertainty as:
A parameter, associated with the result of a measurement, that characterises the dispersion of the values that could reasonably be attributed to the measurand.
Uncertainty is a measure of the spread of the distribution of possible values.
Standard uncertainty describes the standard deviation of the probability distribution describing the spread of possible values
Expanded uncertainty is the standard uncertainty multiplied by a coverage factor, k. The coverage factor is chosen to obtain a desired level of confidence. Most commonly a 95 % confidence interval is chosen. For a Gaussian distribution this is achieved with a coverage factor k = 2. (Note that strictly this provides a 95.45 % confidence interval).
The unknown difference between the measured value and the (unknown) true value. The error is a specific draw from the probability distribution function described by the uncertainty
An offset (additive) or scaling factor (multiplicative) that affects all measurements by a particular instrument. The bias may be estimated, in which case it can be corrected for (a correction), or may be an unknown error.
An adjustment made to correct for a known bias. Note that even after correction there will always be a residual, unknown error.
Effects that for a particular measurement process vary from measurement to measurement. These produce random errors which are entirely uncorrelated and generally are reduced by averaging.
Effects for a particular measurement process that do not vary from measurement to measurement and therefore produce systematic errors that cannot be reduced by averaging.
Aqualitative term describing the spread of obtained measured values. A highprecision data set has small uncertainties associated with random effects.This says nothing about uncertainties associated with systematic effects.Note any quantitative information is provided in the associated uncertainty.
A qualitative term describing the (lack of) systematic uncertainties. A measurement said to be “higher accuracy” would have smaller uncertainties associated with systematic effects. Note that it is possible to have a high accuracy measurement in the presence of large random effects as long as sufficient data is averaged.
Type A evaluation of uncertainty
The GUM distinguishes Type A and Type B methods for evaluating uncertainty. A type A method uses statistical analysis of repeated observations. Usually this is used to estimate the uncertainty associated with random effects. It is possible to use type A methods to estimate the uncertainty associated with effects that are systematic for the measurement of interest but consciously randomised for the purposes of uncertainty evaluation (e.g. by realigning an instrument that would normally not be realigned, or varying a temperature that would normally be constant). In Earth Observation Type A methods are generally used to estimate noise statistics – a random effect process.
Type B evaluation of uncertainty
The GUM describes this as using “other methods”. This may include prior knowledge (e.g. from a calibration certificate or the behaviour of similar instruments), it may include performing theoretical modelling.
An uncertainty given in the same unit as the measured value. This is generally written, as the standard uncertainty, !! or !! .
An uncertainty given in relative units (per cent, parts per million, fractions, etc). This is generally written !!
The process of experimentally obtaining a result. The act of measuring.
The quantity that is being measured (e.g. radiance, reflectance, temperature)
The number, unit and uncertainty of a measurand that comes from measurement
The number and unit obtained from a measurement of a measurand.
Geo-location (or navigation)
This term refers to the process by which the geographical coordinates (e.g., latitude and longitude) of each satellite measurement are determined. The precise determination of geographical coordinates requires information on the satellite orbit, the satellite attitude parameters, and the Geoid. In the case of absence of this information, geographical coordinates are often determined by techniques that use landmarks and control points. The result of geo-location is a geo-referenced satellite measurement without a change in the original geometry of the measurement.
This term refers to satellite measurements that have been geo-located or navigated.
This term refers to the process by which a geo-located or navigated satellite measurement is transformed into the grid of a known coordinate system or type of projection. This process requires interpolation techniques such as cubic-spline or nearest-neighbour. Geo-rectification results in gridded satellite measurements. Geo-rectification is synonymous to gridding for satellite measurements.
This term refers to a systematic transformation of the latitudes and longitudes of locations on the surface of a sphere or an ellipsoid into locations on a plane. Different transformations have been developed and they vary in terms of the priorities they assign on the conservation of angles, area, or distance and on the region of the globe they are optimized for. Projection results in gridded satellite measurements in a specific type of projection.
This term refers to the process of transforming the information represented in one type of projection into another type of projection.
This term refers to the process which assigns a geo-referenced satellite measurement to the appropriate cell in a predefined grid. This step is used to aid the visualisation of satellite imagery as a map in which one grid-cell can be interpreted as one image pixel. Gridding results in gridded satellite measurements. Gridding is synonymous to geo-rectification for satellite measurements.
This term refers to the process of transforming the information represented in one grid into another grid.
This term refers to the data that a satellite collects by scanning the area below its current location, i.e., the swath or the width of this area perpendicular to the satellite’s flight direction.
The CEOS naming convention concerning satellite data processing is based on a nomenclature defined by NASA (Table 1) in 1996:
Table 1 : NASA-EOSDIS satellite data processing levels
|Data Level||NASA-EOSDIS Definition|
|Level 0||Reconstructed, unprocessed instrument and payload data at full resolution, with any and all communications artifacts (e.g., synchronization frames, communications headers, duplicate data) removed.|
|Level 1A||Reconstructed, unprocessed instrument data at full resolution, time-referenced, and annotated with ancillary information, including radiometric and geometric calibration coefficients and georeferencing parameters (e.g., platform ephemeris) computed and appended but not applied to Level 0 data.|
|Level 1B||Level 1A data that have been processed to sensor units (not all instruments have Level 1B source data).|
|Level 2||Derived geophysical variables at the same resolution and location as Level 1 source data.|
|Level 3||Variables mapped on uniform space-time grid scales, usually with some completeness and consistency.|
|Level 4||Model output or results from analyses of lower-level data (e.g., variables derived from multiple measurements).|
Table 2: CEOS satellite data processing levels
|Data Level||CEOS Definition|
|Level 0||Reconstructed unprocessed instrument data at full space time resolution with all available supplemental information to be used in subsequent processing (e.g., ephemeris, health and safety) appended.|
|Level 1||Unpacked, reformatted level 0 data, with all supplemental information to be used in subsequent processing appended. Optional radiometric and geometric correction applied to produce parameters in physical units. Data generally presented as full time/space resolution. A wide variety of sub level products are possible.|
|Level 2||Retrieved environmental variables (e.g., ocean wave height, soil moisture, ice concentration) at the same resolution and location as the level 1 source data.|
|Level 3||Data or retrieved environmental variables which have been spatially and/or temporally re-sampled (i.e., derived from level 1 or 2 products). Such re-sampling may include averaging and compositing.|
The CEOS naming convention provided in Table 2 is clearly inspired by the NASA one, but it proposes a data resampling starting at level 1, and the data can be expressed in Physical units and not only “Sensor units”. The CEOS table does not detail the sub-levels (1A, 2B…). These processing levels are often adapted according to the type of instruments, e.g. for SMOS or Sentinel-1 as a result of different type of acquisition modes.
These definitions provide information on the type of processing do not grant a uniform and consistent processing during the lifetime of a mission as required by the generation of a FCDR. As FCDR usually involve a series of instruments with potentially changing measurement approaches, it is necessary to account for this new constraint in defining processing levels.
Within FIDUCEO, processing levels definition needs to be compatible with the generation of FCDR and CDR. It is suggested to stay as close as possible to the existing data processing level definition but to be more specific on the units and projection.
For the FCDR, the following processing levels are recommended:
Table 3: FIDUCEO satellite data processing levels
|Data Level||FCDR/CDR||FIDUCEO Definition|
|Level 1B||FCDR||Level 1A data that have been processed to sensor units and contains acquisition time pixel location with associated uncertainties. Data processing is performed in a consistent manner for the entire data set.|
|Level 1C||FCDR||Level 1B data that have been georeferenced in a standard grid specific to the instrument (e.g., geostationary projection)|
|Level 1P||FCDR||Level 1B data that have been georeferenced in a projection which is not specific to the instrument.|
|Level 2||CDR||Derived geophysical variables at the same resolution and location as Level 1 source data.|
|Level 3||CDR||Variables mapped on uniform space-time grid scales, usually with some completeness and consistency.|
For Level 1B, 1C and 1P, it is important to also specify the units in which the data are provided. Three different units are suggested for FIDUCEO.
- Radiance [R]: This is the natural “satellite unit”.
- Brightness temperature [T]: This unit, expressed in K, is the most useful unit in IR measurement.
- Bidirectional Reflectance Factor [F]: This unit is the most commonly use unit for observations acquired in the solar spectral region, ie, from 300 to 2000nm.
It is suggested to add the letter R, T or F to the data processing Level 1B, C and P to denote the unit, e.g., Level 1CF. In addition to this main information, FCDR Level 1 data should provide the following information for each pixel:
- The time of acquisition;
- The latitude and longitude;
- The sun and viewing angles;
- The measurement uncertainties;
- The sensor spectral response.
A record of calibrated, geolocated, directly measured satellite observations in geophysical units (such as radiance) in which estimates of total uncertainty (or error covariance) and/or dominant components of uncertainty (or error covariance) are provided or characterised at pixel-level.
Simultaneous Nadir Overpass – a location on the planet where the nadir tracks of two satellites intersect within a given spatial and time distance. Both distance measures can vary, depending on the context.
A sensor “point” measurement that is matched with another sensor’s “point” measurement sufficiently close in space and time. In a more practical context, a match up consists of two or more sensor pixels (with optional surrounding pixels) that have been acquired at the same time and cover the same location. Time and spatial difference allowed are defined by the scientific aim. The match up data consists of all data variables at the location and all metadata for each sensor.
Kidder, S. Q. , and Von der Haar, T. H., 1995, Satellite Meteorology (San Diego: Academic Press). Pages 157ff.
Rao, P. K., Holmes, S. J., Anderson, R. K., Winston, J. S. and Lehr, P. E., 1990. Weather satellites: Systems data and environmental applications American meteorological society (Boston: American Meteorological Society). Pages 481ff.