by Ralf Quast and Ralf Giering Recently, Emma Wooliams has explained how the FIDUCEO project performs recalibration of satellite data series to produce new harmonised fundamental climate data records from raw counts. The harmonisation process involves refitting the calibration parameters, taking into account all error covariance. Also recently, Yves Govaerts has exemplified how FIDUCEO will derive new thematic climate data records and has pointed out the use of a rigorous uncertainty propagation scheme as an innovative key task. The Guide to the expression of Uncertainty in Measurement (GUM) [1] has formalised a recommended uncertainty propagation scheme. For instance, let x1 , x2 be measured quantities and let Cx denote their error covariance matrix. Let further y1, …, ym denote some variables derived from these measured quantities. Then the GUM states that the error covariance matrix of the derived quantities is given by the matrix product
The row vectors of the Jacobian matrix Jyx are the transposed gradients of the variables y1, …, ym with respect to the measured quantities x. In general, the error covariance matrix of the derived variables is not diagonal, even if the error covariance matrix of the measured quantities is. The variables in a thematic climate data record (CDR) are derived from variables in a fundamental CDR (brightness temperature, radiance, reflectance) by means of a retrieval algorithm. The retrieval algorithm itself may use a certain set of additional parameters, too. Now putting the above uncertainty propagation scheme into the CDR context, the fundamental CDR variables and the set of algorithm parameters correspond to the measured quantities x, while the thematic CDR variables correspond to the derived quantities y. Assuming the error covariance matrix of the measured quantities is known, the main difficulty in applying the GUM scheme is to compute the Jacobian matrix of partial derivatives. Retrieval algorithms often consist of complex numerical code that involves radiative transfer calculations and iterative equation solving. Manually coding derivatives is usually not feasible, and if feasible, time consuming and prone to mistakes. Numerical differentiation is simple to implement, but scales poorly for gradients and is very inaccurate due to round-off and truncation errors. Symbolic differentiation requires the retrieval algorithm to be expressed as a closed-form mathematical formula, ruling out algorithmic control flow and severely limiting expressivity. A very powerful fourth technique, Algorithmic differentiation (AD), works by systematically applying the chain rule of differential calculus at the elementary programming language operator level [2, 3]. AD allows the accurate evaluation of derivatives at machine precision, with only a small constant factor of overhead and ideal asymptotic efficiency. In contrast with the effort involved in arranging code into closed-form expressions for symbolic differentiation, AD can often be applied to existing source code with minimal change. An example of an advanced AD source-to-source compiler is Transformation of Algorithms in Fortran (TAF) [4]. Because of its generality, TAF is an already established tool in applications including Earth system modelling [5], bio-geochemical models [6], data assimilation [7, 8], sensitivity analysis [9], radiative transfer models [10], aerodynamics [11], and atmospheric chemistry and physics [12]. A demonstrator is available online. Once computed, the covariance matrix of the CDR variables can be included in the CDR or be used to generate an ensemble CDR. Covariance elements may often be larger than variance elements and hence the provision and use of covariance information in a CDR is essential.

References

[1] Joint Committee for Guides in Metrology. 2008. “Guide to the Expression of Uncertainty in Measurement.” [2] Griewank, A., A. Walther. 2008. “Evaluating Derivatives. Principles and Techniques of Algorithmic Differentiation.” SIAM. DOI: 10.1137/1.9780898717761 [3] Giering, R., T. Kaminski. 1998. “Recipes for Adjoint Code Construction.” ACM Trans. Math. Soft. 24 (4): 437–474. DOI: 10.1145/293686.293695 [4] Giering, R., T Kaminski. 2003. “Applying TAF to Generate Efficient Derivative Code of Fortran 77-95 Programs.” Proc. Appl. Math. Mech. 2 (1): 54–57. DOI: 10.1002/pamm.200310014 [5] Blessing, S., T. Kaminski, F. Lunkeit, I. Matei, R. Giering, A. Köhl, M. Scholze, P. Herrmann, K. Fraedrich, D. Stammer. 2014. “Testing Variational Estimation of Process Parameters and Initial Conditions of an Earth System Model.” Tellus A 66, 22606. DOI: 10.3402/tellusa.v66.22606 [6] Giering, R. 2000. “Tangent Linear and Adjoint Biogeochemical Models.”, in Inverse Methods in Global Biogeochemical Cycles (eds P. Kasibhatla, M. Heimann, P. Rayner, N. Mahowald, R. G. Prinn, D. E. Hartley). American Geophysical Union, Washington, DC. DOI: 10.1029/GM114p0033 [7] Kaminski, T., W. Knorr, G. Schürmann, M. Scholze, P. J. Rayner, S. Zaehle, S. Blessing, W. Dorigo, V. Gayler, R. Giering, N. Gobron, J. P. Grant, M. Heimann, A. Hooker-Strout, S. Houweling, T. Kato, J. Kattge, D. Kelley, S. Kemp, E. N. Koffi, C. Köstler, P.P. Mathieu, B. Pinty, C. H. Reick, C. Rödenbeck, R. Schnur, K. Scipal, C. Sebald, T. Stacke, A. Terwisscha van Scheltinga, M. Vossbeck, H. Widmann, T. Ziehn. 2013. “The BETHY/JSBACH Carbon Cycle Data Assimilation System: Experiences and Challenges.” J. Geophys. Res. 118 (4): 1414–1426. DOI: 10.1002/jgrg.20118 [8] Stammer D., C. Wunsch, R. Giering, C. Eckert, P. Heimbach, J. Marotzke, A. Adcroft, C. N. Hill, J. Marshall. 2002. “The Global Ocean Circulation During 1992-1997, Estimated from Ocean Observations and a General Circulation Model.” J. Geophys. Res. 107 (C9): 1-1–1-27. DOI: 10.1029/2001JC000888 [9] Marotzke, J., R. Giering, Q. K. Zhang, D. Stammer, C. N. Hill, T. Lee. 1999. “Construction of the Adjoint MIT Ocean General Circulation Model and Application to Atlantic Heat Transport Sensitivity.” J. Geophys. Res. 104 (C12): 29529–29547. DOI: 10.1029/1999JC900236 [10] Voßbeck, M., M. Clerici, T. Kaminski, B. Pinty, T. Lavergne, R. Giering. 2010. “An Inverse Radiative Transfer Model of the Vegetation Canopy Based on Automatic Differentiation. “. Inverse Problems 26 (095003): 1–15. DOI: 10.1088/0266-5611/26/9/095003 [11] Giering, R., T. Kaminski, T. Slawig. 2005. „Generating Efficient Derivative Code with TAF: Adjoint and Tangent Linear Euler Flow Around an Airfoil.” Future Generation Computer Systems 21 (8): 1345–1355. DOI:10.1016/j.future.2004.11.003 [12] Henze, D. K., A. Hakami, J. H. Seinfeld. 2007. “Development of the adjoint of GEOS-Chem.” Atmos. Chem. Phys. 7: 2413–2433. DOI: 10.5194/acp-7-2413-2007