![\"\"](\"https:\/\/research.reading.ac.uk\/dare\/wp-content\/uploads\/sites\/5\/2017\/09\/Nichols-3.jpg\")
<\/p>\nBy Nancy Nichols<\/p>\n
Should you care about the numerical accuracy of your computer?\u00a0 After all, most machines now retain about 16 digits of accuracy, but usually only about 3 – 4 figures of accuracy are needed for most applications; \u00a0so what\u2019s the worry?\u00a0\u00a0 To demonstrate, there have been a number of spectacular disasters due to numerical rounding error.\u00a0 One of the most well known is the failure of a Patriot Missile to track and intercept an Iraqi Scud missile in Dharan, Saudi Arabia, on February 25, 1991, resulting in the deaths of 28 American soldiers.<\/p>\n
<\/p>\n
The failure was ultimately attributable to poor handling of \u00a0rounding errors.\u00a0 The computer doing the tracking calculations had an internal clock whose values were truncated when converted to floating-point arithmetic with an error of about 2-20 <\/sup>.\u00a0\u00a0 The clock had run up a time of 100 hours, so the calculated elapsed time was too long by 2-20<\/sup> x 100 hours = 0.3433 seconds, during which time a Scud would be expected to travel more than half a kilometer.<\/p>\n <\/p>\n (See The Patriot Missile Failure)<\/p>\n The same problem arises in other algorithms that accumulate and magnify small round-off errors due to the finite (inexact) representation of numbers in the computer. \u00a0\u00a0Algorithms of this kind are referred to as \u2018unstable\u2019 methods.\u00a0 Many numerical schemes for solving differential equations have been shown to magnify small numerical errors.\u00a0 It is known, for example, that L.F. Richardson\u2019s original attempts at numerical weather forecasting were essentially scuppered due the unstable methods that were used to compute the atmospheric flow.\u00a0 \u00a0Much time and effort have now been invested in developing and carefully coding methods for solving algebraic and differential equations such as to guarantee stability.\u00a0\u00a0 Excellent software is publicly available.\u00a0 Academics and operational weather forecasting centres in the UK have been at the forefront of this research.<\/p>\n Even with stable algorithms, however, it may not be possible to compute an accurate solution to a given problem.\u00a0\u00a0 The reason is that the solution may be sensitive to small errors\u00a0 –\u00a0 that is, a small error in the data describing the problem causes large changes in the solution.\u00a0 Such problems are called \u2018ill-conditioned\u2019.\u00a0\u00a0 Even entering the data of a problem into a computer\u00a0 – \u00a0for example, the initial conditions for a differential equation or the matrix elements of an eigenvalue problem\u00a0 –\u00a0 \u00a0must introduce small numerical errors in the data.\u00a0 If the problem is ill-conditioned, these then lead to large changes in the computed solution, which no method can prevent.<\/p>\n So how do you know if your problem is sensitive to small perturbations in the data?\u00a0 Careful analysis can reveal the issue, but for some classes of problems there are measures of the sensitivity, or the \u2018conditioning\u2019, of the problem that can be used.\u00a0\u00a0 For example, it can be shown that small perturbations in a matrix can lead to large relative changes in the inverse of the matrix if the \u2018condition number\u2019 of the matrix is large.\u00a0 The condition number is measured as the product of the norm of the matrix and the norm of its inverse.\u00a0 Similarly\u00a0 small changes in the elements of a matrix will cause its eigenvalues to have large errors if the \u2018condition number\u2019 of the matrix of eigenvectors is large.\u00a0\u00a0 Of course to determine the condition numbers is a problem implicitly, but accurate computational methods for estimating the condition numbers are available .<\/p>\n An example of an ill-conditioned matrix is the covariance matrix associated with a Gaussian distribution.\u00a0\u00a0 The following figure shows the condition number of a covariance matrix obtained by taking samples from a Gaussian correlation function at 500 points, using a step size of 0.1, for varying length-scales [1]. This result is surprising and very significant for numerical weather prediction (NWP) as the inverse of covariance matrices are used to weight the uncertainty in the model forecast and in the observations used in the analysis phase of weather prediction.\u00a0 The analysis is achieved by the process of data assimilation, which combines a forecast from a computational model of the atmosphere with physical observations obtained from in situ and remote sensing instruments.\u00a0 If the weighting matrices are ill-conditioned, then the assimilation problem becomes ill-conditioned also, making it difficult to get an accurate analysis and subsequently a good forecast [2].\u00a0 Furthermore, the worse the conditioning of the assimilation problem becomes, the more time it takes to do the analysis. This is important as the forecast needs to be done in \u2018real\u2019 time, so the analysis needs to be done as quickly as possible.<\/p>\n One way to deal with an ill-conditioned system is to rearrange the problem to so as to reduce the conditioning whilst retaining the same solution.\u00a0 A technique for achieving this is to \u2018precondition\u2019 the problem using a transformation of the variables.\u00a0 This is used regularly in NWP operational centres with the aim of ensuring that the uncertainties in the transformed variables all have a variance of one [1][2].\u00a0 In this table we can see the effects of the length-scale of the error correlations in a data assimilation system [1]\u00a0 S.A Haben. 2011. Conditioning and Preconditioning of the Minimisation Problem in<\/p>\n Variational Data Assimilation, University of Reading, Department of Mathematics and Statistics, Haben PhD Thesis<\/a><\/p>\n![\"\"](\"https:\/\/research.reading.ac.uk\/dare\/wp-content\/uploads\/sites\/5\/2017\/09\/Nichols.png\")
The condition number increases rapidly to 107<\/sup> for length-scales of only size \u00a0L = 0.2\u00a0 and, for length scales larger than 0.28, the condition number is larger than the computer precision and cannot even be calculated accurately.<\/p>\n![\"\"](\"https:\/\/research.reading.ac.uk\/dare\/wp-content\/uploads\/sites\/5\/2017\/09\/Nichols-2.png\")
on the number of iterations it takes to solve the problem, with and without preconditioning of the problem [1].\u00a0 The conditioning of the problem is improved and the work needed to solve the problem is significantly reduced. \u00a0So checking and controlling the conditioning of a computational problem is always important!<\/p>\n