Aerosols from AVHRR case study

By Thomas Popp

In this case study we show how useful sophisticated uncertainties of FIDUCEO easyFCDR AVHRR Level1b datasets (split into 3 major components with different correlation structures) are for propagating them to uncertainties of aggregated products (in this case Aerosol Optical Depth). We apply methodologies and tools developed in FIDUCEO to analyse the propagation of uncertainties through a retrieval algorithm in an efficient manner.

Introduction: AOD from AVHRR

The AVHRR instrument, which was designed for cloud and land observations is a weak instrument if it is used for the retrieval of a Climate Data Record (CDR) of Aerosol Optical Depth (AOD) over land, because of its small information content with practically only one useful channel with uncertain calibration. However, AVHRR offers a long historic record back to 1978 from the series of instruments flown on NOAA and METOP platforms.

The measurement equation shows how AOD can be inverted from top-of-atmosphere (TOA) reflectance measurements in the red band, (directional) surface albedo estimated from the mid-infrared channel at 3.7 µm and by assuming optical properties of atmospheric aerosol (aerosol type) – note the colour coding used throughout this description to identify the dominant three effects:

AOD_{630}=g(\textcolor{red}{R_{ 630\atop TOA}};\theta_{S},\theta_{0},\theta_{\varphi}; \textcolor{orange}{Alb_{surf}},\textcolor{green}{aerosol_{type}})+\textcolor{blue}{0}

with AOD_{630}  the resulting Aerosol Optical Depth at 630 nm,

 g the retrieval operator,

 \textcolor{red}{R_{ 630\atop TOA}} the input top-of-atmosphere reflectance at 630 nm,

 \theta_{S},\theta_{0},\theta_{\varphi} the observation angles (sun and observer zenith, relative azimuth)

\textcolor{orange}{Alb_{surf}} the (directional) surface albedo

\textcolor{green}{aerosol_{type}} a combination of aerosol optical properties

 \tau^{trace gases}_{i} the product of (weak) transmissions by several trace gases

 h_{surf} the surface elevation

aerosol_{profile} the vertical profile of aerosol extinction

A demonstration was implemented, where a regional AVHRR AOD CDR of 10 years (2003 – 2012) over land covering Europe and Northern Africa was produced. \textcolor{red}{R_{ 630\atop TOA}}  is the measured AVHRR Channel 1 reflectance; \textcolor{orange}{Alb_{surf}} is estimated from the reflectance part of the AVHRR Channel 3B (using a linear conversion determined by the vegetation index). The \textcolor{green}{aerosol_{type}} is provided by a climatology (1 degree lat / lon, monthly) of mixing factors of four basic aerosol components (fine mode weakly absorbing, fine mode strongly absorbing, desert dust, sea salt); the climatology was derived from a median of global aerosol models (AEROCOM).  Look-up tables (LUT) of radiative transfer calculations are stored as second order polynomials for each of 36 aerosol mixtures representing a realistic range of true atmospheric aerosol compositions.

Figure 1 shows the FIDUCEO traceability chain of this processing. AOD is retrieved in the red band (630 nm, AVHRR Channel 1) for selected single “dark pixels” (internal L2A product) and then aggregated to super pixels (3 x 3) of about 12 x 12 km² (L2B product); additionally gridded products (L3) on 1 degree latitude / longitude are produced by averaging all super-pixels per day and then all daily values during a month. The main input is from AVHRR Channel 1, but also the other channels are needed for calculating NDVI, estimating surface albedo and for cloud mask tests. Dark pixels are determined (upper left branch in the block diagram) by cloud masking (to avoid cloud contamination) and by filtering all cloud-free pixels for (partial) vegetation cover and for darkness in the mid-infrared band. AOD is retrieved by inversion according to the measurement equation (central right branch in the block diagram). In order to model the dependence of the results to different aerosol types, the retrieval is repeated for an ensemble of 36 realizations.

Figure 1: Traceability chain for AVHRR AOD CDR (from “FIDUCEO Uncertainty report for AVHRR AOD CDR”)

Estimating AOD uncertainties for single pixels

Estimating uncertainties of the retrieved AOD on pixel level  is crucial to understand the reliability of the results. This is particularly important, since the sensitivity of the retrieved AOD to the measured signal varies largely with retrieval conditions (AOD itself, surface brightness, aerosol optical properties / aerosol type, observing geometry). Given the weak retrieval from AVHRR, we choose a pragmatic approach for the estimation of pixel-level AOD uncertainties, which is based on lessons learned during the ESA Aerosol_cci project and focuses on the uncertainty equation with dominant terms (also called effects):

u(AOD)= \sqrt{\textcolor{red}{\frac{\partial AOD}{\partial R_{TOA}}u(R_{TOA})})^2+\textcolor{orange}{\frac{\partial AOD}{\partial Alb_{surf}}u(Alb_{surf})})^2+(\textcolor{green}{u(AOD)_{ensemble}})^2}+\textcolor{blue}{u^2(0)}

with u(AOD) the AOD uncertainty

\textcolor{red}{\frac{\partial AOD}{\partial R_{TOA}}}the sensitivity of AOD to \textcolor{red}{R_{TOA}}

 \textcolor{red}{u(R_{TOA})} the uncertainty of \textcolor{red}{R_{ToA}}

\textcolor{orange}{\frac{\partial AOD}{\partial Alb_{surf}}} the sensitivity  of AOD to \textcolor{orange}{Alb_{surf}}

 \textcolor{orange}{u(Alb_{surf})} the uncertainty of \textcolor{orange}{Alb_{surf}}

\textcolor{green}{u(AOD)_{ensemble}}  the spread of an ensemble of different aerosol types

 \textcolor{blue}{u^2(0)} the sum of weaker uncertainties, which are considered significantly smaller or can be assumed to be fully independent, so that they average out on larger spatial / temporal scales

Neglected uncertainties summarized in the term \sigma^2(0)  do contain trace gas absorption correction (small due to setup of window channels), altitude dependent Rayleigh scattering correction, vertical layering of AOD (both small in the red band), look-up table errors versus full radiative transfer calculations, including interpolation errors between distinct angular values (both proven with full radiative transfer calculations), and interpolation values between distinct aerosol types.

The uncertainty equation shows that the uncertainties in the AOD CDR do not only originate from propagation of uncertainties in measured reflectances, but also assumptions, simplifications, and lacking knowledge in the retrieval do add major contributions.

To derive AOD uncertainties the following FIDUCEO analysis tree for the single pixel AOD inversion (L2A) was made (Figure 2), which determines how each of the uncertainty components can be calculated. In this diagram the calculation of the uncertainty of each of the three dominant terms of the right side of the measurement function in the center is depicted. The (\textcolor{red}{red}) uncertainty of the reflectance inversion is derived as product of the reflectance uncertainty \textcolor{red}{u(R_{TOA})}  and the AOD sensitivity to the measured reflectance . Also the (\textcolor{orange}{orange}) uncertainty of the surface albedo is calculated as product of the albedo uncertainty \textcolor{orange}{u(Alb_{surf})}  and the AOD sensitivity to the albedo \textcolor{orange}{\frac{\partial AOD}{\partial Alb_{surf}}} . However, in this case the albedo uncertainty needs to be calculated by propagating the uncertainties of NDVI (and ultimately R₆₃₀ and R₈₇₀) and R_3.7 through the linear conversion function used. Furthermore, a constant global uncertainty value of 0.01 is added which reflects the uncertainty of using the linear regression. The aerosol type uncertainty (\textcolor{green}{green}) cannot be calculated with a similar product, but this is replaced by the spread of an ensemble of 36 different aerosol mixtures.

Those uncertainties are calculated based on the reflectance uncertainties contained in the FIDUCEO easyFCDR AVHRR L1B product. This easyFCDR product provides three separate uncertainty components for each channel reflectance (or brightness temperature):

• common (globally fully correlated uncertainties)
• indpendent (random, globally uncorrelated)
• structured (correlated along defined distances,with correlation length and function

Each component is propagated separately and at the end of the L2A processing all contributions with the same correlation structure (i. e. all common, all independent, all structured) are summed up (according to GUM as square root of the squared contributions).

Figure 2: Measurement-function centered analysis tree for AVHRR AOD CDR: L2A processing

Propagating AOD uncertainties

The propagation of uncertainties from L2A to L2B (and similar to L3) is then determined by the FIDUCEO analysis tree (L2B, Figure 3). Here the correlation structures are now taken into account. The independent contributions (no correlation at all) can be squared and thus this noise term is reduced by 1/\sqrt{N} with increasing number of pixels N. In contradiction, the common contributions are simply averaged and achieve no reduction with growing number N. In between those two extremes, the structured contributions depend on the correlation c_ij (which typically decrease with growing distance of elements I and j). In the end, the total super-pixel uncertainty is then summed up from these three parts (squared, as they are independent from each other).

One particular element of the super-pixel uncertainty is the contribution due to uncertainties of the cloud masking. The cloud retrieval algorithm used results in a Bayesian cloud probability, so that we can derive two different cloud masks (weak and strong) by defining two different probability thresholds (5% and 50%). The AOD retrieval is then used for all cloud-free pixels of either cloud mask and the average AOD per super-pixel cell is calculated. The AOD difference between conservative and relaxed cloud masking is then used as proxy for the cloud mask induced uncertainty.

Figure 3: Measurement-function centered analysis tree for AVHRR AOD CDR: L2B processing

In propagating the structured contributions, the correlation functions of uncertainties (their probability distribution form and their length scale) in space and time dimension need to be known or estimated – note that this is not the correlation of the physical quantities but the correlation of their uncertainties. This choices identified in FIDUCEO are shown in Table 1. Note that this information is directly used from the L1B product for the reflectance and surface albedo effects, while it needs to be estimated from physical understanding aerosol plumes and cloud systems.

Table 1: Averaging the different effects to super-pixels and gridded products

Effect	Uncertainty correlation structure	Spatial correlation Daily gridded data	Temporal correlation Monthly gridded data
TOA reflectance	Common within line Structured across lines Uncorrelated in time	pdf from lv1b (R0.63)	–
Surface albedo	Common within line independent across lines Uncorrelated in time	pdf from lv1b (R3.7, NDVI)	–
Aerosol type	Climatology grid Typical aerosol lifetime	1° / rectangular	1 week / rectangular
Cloud mask	None (extremely short cloud lifetime)	–	–

One orbit case study

To illustrate the uncertainty calculation, step-by-step results for one orbit (NOAA-18, 16.08.2008 over Central Europe) are shown here. During this day heavy fire activity happened in Eastern Europe (as proven by AATSR satellite images, not shown here), from which aerosols were transported to Central Europe. Figure 4 shows an obvious cloud system (orange / yellow) and areas with broken clouds (e.g. over Germany and Western Scandinavia) in the visible image (red band). Inverted AOD and its total uncertainty are also shown together with the selected dark pixels used for the inversion.

Figure 4:Example scene (NOAA-18, 16.08.2008): upper left: retrieved AOD; upper right: total AOD uncertainty; lower left: selected dark fields (yellow: from strict cloud masking, red: from weak cloud masking, blue: cloud-free but not used); lower right: top of atmosphere reflectance in the AVHRR red band.

The four dominant uncertainty contributions are shown in Figure 5 due to the reflectance inversion, due to the estimated surface albedo, due to the weak knowledge of the aerosol mixture, due to cloud masking uncertainties (note the different scales).

Figure 5: Example scene (NOAA-18, 16.08.2008): Dominant uncertainty contributions, from upper left to lower right: reflectance term, albedo term, aerosol type term, cloud mask term.

Calculating the cloud mask induced uncertainty is shown in the Figure 6. The difference of retrieved AIOD maps based on two different cloud masks (with two different cloud probability thresholds) is exploited to estimate the cloud-mask induced uncertainty. The two probability threshold values have been determined experimentally by testing several values, so that sufficient coverage can be achieved while the main effects due to broken clouds and cloud edges can be seen. Note that the resulting AOD and all other uncertainties are calculated on the basis of the more conservative (i.e. safer) cloud probability threshold.

Figure 6: Example scene (NOAA-18, 16.08.2008): cloud mask induced AOD uncertainties, from upper left to lower right: average AOD with strict cloud filtering; average AOD with weak cloud filtering, cloud probability retrieved, difference of the AOD maps.

In Figure 7, the propagation of AOD uncertainties with different correlation structures is illustrated. While common uncertainties undergo no reduction at all, the independent (random) uncertainties are subject to a major noise reduction.

Figure 7: Example scene (NOAA-18, 16.08.2008): Propagating uncertainty components from pixels (1×1, left) to super-pixels (3×3, right) for different correlation structures: common component (upper line), independent component (lower line).

The propagation of total AOD uncertainties to super-pixels (3×3) and to gridded values (1 degree, with only one orbit as input) is shown in Figure 8. Clearly, uncertainties get reduced by the averaging, but not everywhere the reduction is the same. Finally, the resulting total AOD uncertainty is compared to imaginative total uncertainties, which would come out if all uncertainties were considered fully independent (“assuming all random”) or fully correlated (“assuming all correlated”). Obviously, the uncertainties propagated with the wealth of the FIDUCEO L1B uncertainty correlation structures fall between the two extremes while the effect of noise reduction differs in different conditions (AOD, surface brightness, geometry, cloudiness).

Figure 8: Example scene (NOAA-18, 16.08.2008): AOD uncertainty on super-pixel (3×3) resolution (upper left) and daily gridded (upper right, with one orbit input only). Fictional 3×3 uncertainties are shown in the lower line right), are fictionally assuming no / random (left) or full correlation (right).

Conclusions

A case study is shown to illustrate with one orbit of AVHRR data the propagation of different uncertainty components with different correlation structures from the single pixel inversion to super-pixels and daily gridded values. The (simple) demonstration algorithm with its uncertainty propagation to L2B and L3 datasets was applied to 10 years of AVHRR data over Europe and North Africa (2003 – 2005 from NOAA-16 and 2005 – 2012 from NOAA-18) and its results are available on the FIDUCEO legacy website.

The main advantage of this approach is the availability of detailed L1B uncertainties in the FIDCUEO easyFCDR format which contain separate components for common, independent and structured uncertainties. Furthermore, the FIDUCEO methodology and tools helped to structure and guide the analysis of the uncertainty propagation. All these allow detailed propagation through the different aggregation levels where the correlation structures can be fully taken into account. The uncertainties thus estimated depend on the propagated effects of the measured reflectances, but also on assumptions and simplifications in the inversion approach.