{"id":2625,"date":"2025-04-03T10:06:53","date_gmt":"2025-04-03T09:06:53","guid":{"rendered":"https:\/\/research.reading.ac.uk\/met-darc\/?p=2625"},"modified":"2025-04-13T19:55:38","modified_gmt":"2025-04-13T18:55:38","slug":"the-least-squares-method-in-data-assimilation","status":"publish","type":"post","link":"https:\/\/research.reading.ac.uk\/met-darc\/2025\/04\/03\/the-least-squares-method-in-data-assimilation\/","title":{"rendered":"The least squares method in data assimilation"},"content":{"rendered":"<p>by Ross Bannister, April 2025<\/p>\n<p>The method of least squares (or MLS, Barlow, 1993) is at the very centre of variational data assimilation (or \u201cVar\u201d, Schlatter, 2000). Var is one of the major data assimilation (DA) approaches used in numerical weather prediction (NWP) to estimate a model\u2019s state from observations. The basic idea of Var is to find a model\u2019s state \u2013 let\u2019s call this <strong>x<\/strong> \u2013 that agrees as closely as possible with observations \u2013 let\u2019s call these <strong>y<\/strong>. The invention of the MLS is credited to mathematicians Legendre and Gauss near the turn of the 19th century.<\/p>\n<h4>The basic principle of least squares<\/h4>\n<p>The MLS relies on predicting what the observations \u2018ought\u2019 to be given the model\u2019s state. In DA terminology this uses the \u2018observation operator\u2019, often written mathematically as <em>H<\/em>(<strong>x<\/strong>). The predicted observations will be different from the actual (measured) observations. The MLS poses the question: what <strong>x<\/strong> minimises the objective function \u00bd|<strong>y<\/strong> &#8211; <em>H<\/em>(<strong>x<\/strong>)|<sup>2<\/sup>, where | |<sup>2<\/sup> takes the sum of the squares of the differences between elements of <strong>y<\/strong> and <em>H<\/em>(<strong>x<\/strong>).<\/p>\n<p>Imagine we have a simple model with one grid box and two variables: T<sub>0<\/sub> (starting temperature) and d<sub>t<\/sub>T (how fast the temperature changes over time). We also have observations of temperature at the grid box at times in the future, namely <em>y<\/em><sub>1<\/sub> = <em>T<\/em><sup>ob<\/sup><sub>1<\/sub> at <em>t<\/em> = <em>t<\/em><sub>1<\/sub>, <em>y<sub>2<\/sub><\/em> = <em>T<\/em><sup>ob<\/sup><sub>2<\/sub> at <em>t<\/em> = <em>t<\/em><sub>2<\/sub>, etc. We ask the question: <em>what values of<\/em> <em>T<\/em><sub>0<\/sub> and <em>d<sub>t<\/sub>T<\/em> <em>best match the observations<\/em>? The prediction of the first measured temperature is <em>H<\/em><sub>1<\/sub>(<em>T<\/em><sub>0<\/sub>, <em>d<sub>t<\/sub>T<\/em>) = <em>d<sub>t<\/sub>T<\/em><em>\u00d7t<sub>1<\/sub> + T<sub>0<\/sub><\/em>, and similarly for the remaining observations. Readers who are familiar with the equation of a straight line \u201c<em>y<\/em>=<em>mx<\/em>+<em>c<\/em>\u201d may see parallel with our observation operator. It is therefore instructive to represent the observation operator as a straight line with gradient <em>d<sub>t<\/sub>T<\/em> and intercept <em>T<sub>0<\/sub><\/em>. See the Fig 1. for the graphical interpretation, where the best fit <em>d<sub>t<\/sub>T<\/em> and <em>T<sub>0<\/sub><\/em> are found.<\/p>\n<p>Figure: Movie illustration of the method of least squares to find the two parameters dtT and T0 from observations of T.<\/p>\n<div style=\"width: 640px;\" class=\"wp-video\"><video class=\"wp-video-shortcode\" id=\"video-2625-1\" width=\"640\" height=\"828\" preload=\"metadata\" controls=\"controls\"><source type=\"video\/mp4\" src=\"https:\/\/research.reading.ac.uk\/met-darc\/wp-content\/uploads\/sites\/48\/2025\/04\/LS-ezgif.com-gif-to-mp4-converter.mp4?_=1\" \/><a href=\"https:\/\/research.reading.ac.uk\/met-darc\/wp-content\/uploads\/sites\/48\/2025\/04\/LS-ezgif.com-gif-to-mp4-converter.mp4\">https:\/\/research.reading.ac.uk\/met-darc\/wp-content\/uploads\/sites\/48\/2025\/04\/LS-ezgif.com-gif-to-mp4-converter.mp4<\/a><\/video><\/div>\n<h4>Beyond the straight line<\/h4>\n<p>The straight line is often a student\u2019s first taste of the MLS, but least squares is more than that.<\/p>\n<p>Firstly, <strong>x<\/strong> can contain more than two variables. In NWP <strong>x<\/strong> comprises billions of variables, essentially representing the values of multiple physical quantities in each model grid box.<\/p>\n<p>Secondly, the operator <em>H<\/em>(<strong>x<\/strong>) is not limited to linear relationships. There are many instances in NWP where observation predictions are non-linearly related to model variables. Perhaps the simplest is for an observation of the wind speed. Wind speed is related to the east-west (<em>u<\/em>) and north-south (<em>v<\/em>) wind components via \u221a(<em>u<\/em><sup>2<\/sup> + <em>v<sup>2<\/sup><\/em>). Other examples of non-linear components of <em>H<\/em>(<strong>x<\/strong>) are more complex. For example, the prediction of quantities that a satellite would see (radiances, Eyre, 1989), and the NWP model itself when observations are at a later time than the model state to be determined (Pires et al., 1996). Whatever the form of <em>H<\/em>(<strong>x<\/strong>), one can attempt to minimise the objective function using a descent algorithm. However, this is harder when <em>H<\/em>(<strong>x<\/strong>) is non-linear as non-linearity can lead to multiple minima (Pires et al., 1996).<\/p>\n<p>Thirdly, in general not all observations should play an equal role in estimating <strong>x<\/strong>. In a diverse observing system observations will come from instruments with a range of precisions. The MLS can account for instrument precision by weighting the contributions to the objective function according to the inverse square of each observation\u2019s error. This results in low precision observations (high error) having less influence than high precision observations. This is weighted least squares, which can be generalised to ideas around error covariances of the observations and naturally links to ideas of likelihood (maximum likelihood estimates).<\/p>\n<p>Fourthly, the standard MLS accounts for uncertainty in the observations, but not in the time that they are made (<em>t<sub>i<\/sub><\/em>). Accounting for uncertainty in the times adds another layer of complexity to the problem, e.g. Orear, 1982.<\/p>\n<h4>Equifinality and null spaces<\/h4>\n<p>Equifinality is the idea that many situations can lead to the same result. In MLS, this happens when more than one <strong>x<\/strong> leads to the same minimum value of the objective function. When <em>H<\/em>(<strong>x<\/strong>) is linear, this is connected to the concept of a <em>null space<\/em>. Think of <strong>x<\/strong> as existing in a multi-dimensional space. The null space consists of the directions in that space that lead to the same value of the objective function. In NWP, Var problems have a null space as there are far more variables in <strong>x<\/strong> than observations in <strong>y<\/strong>. To solve this we add an extra term to the objective function. The new term penalises deviations from a known first-guess state (the background state, <strong>x<\/strong><sup>b<\/sup>). The objective function becomes \u00bd|<strong>y<\/strong> \u2013 <em>H<\/em>(<strong>x<\/strong>)|<sup>2<\/sup> + \u00bd|<strong>x<\/strong> &#8211; <strong>x<\/strong><sup>b<\/sup>|<sup>2<\/sup> weighting the observations and the background state.<\/p>\n<h4>Conclusions<\/h4>\n<p>Using and understanding the MLS can be complex, and whole books cover the subject, e.g. Lewis et al., 2006. Nevertheless, it remains a powerful and effective tool in DA.<\/p>\n<h4>References<\/h4>\n<p>Barlow RJ, Statistics: a guide to the use of statistical methods in the physical sciences, volume 29, John Wiley &amp; Sons, 1993.<\/p>\n<p>Eyre JR, Inversion of cloudy satellite sounding radiances by nonlinear optimal estimation. I: Theory and simulation for TOVS. Quarterly Journal of the Royal Meteorological Society, 115, 1001-1026, 1989.<\/p>\n<p>Lewis JM, Lakshmivarahan S, and Dhall S, Dynamic data assimilation: a least squares approach, volume 13. Cambridge University Press, 2006.<\/p>\n<p>Orear J, Least squares when both variables have uncertainties. American Journal of Physics, 50, 912-916, 1982.<\/p>\n<p>Pires C, Vautard R, and Talagrand O, On extending the limits of variational assimilation in nonlinear chaotic systems. Tellus A, 48, 96-121, 1996.<\/p>\n<p>Schlatter TW, Variational assimilation of meteorological observations in the lower atmosphere: a tutorial on how it works. Journal of atmospheric and solar-terrestrial physics, 62, 1057-1070, 2000.<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>by Ross Bannister, April 2025 The method of least squares (or MLS, Barlow, 1993) is at the very centre of variational data assimilation (or \u201cVar\u201d, Schlatter, 2000). Var is one&#8230;<a class=\"read-more\" href=\"&#104;&#116;&#116;&#112;&#115;&#58;&#47;&#47;&#114;&#101;&#115;&#101;&#97;&#114;&#99;&#104;&#46;&#114;&#101;&#97;&#100;&#105;&#110;&#103;&#46;&#97;&#99;&#46;&#117;&#107;&#47;&#109;&#101;&#116;&#45;&#100;&#97;&#114;&#99;&#47;&#50;&#48;&#50;&#53;&#47;&#48;&#52;&#47;&#48;&#51;&#47;&#116;&#104;&#101;&#45;&#108;&#101;&#97;&#115;&#116;&#45;&#115;&#113;&#117;&#97;&#114;&#101;&#115;&#45;&#109;&#101;&#116;&#104;&#111;&#100;&#45;&#105;&#110;&#45;&#100;&#97;&#116;&#97;&#45;&#97;&#115;&#115;&#105;&#109;&#105;&#108;&#97;&#116;&#105;&#111;&#110;&#47;\">Read More ><\/a><\/p>\n","protected":false},"author":931,"featured_media":2649,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"__cvm_playback_settings":[],"__cvm_video_id":"","footnotes":"","_links_to":"","_links_to_target":""},"categories":[1],"tags":[57,56,59,58,23],"class_list":["post-2625","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorised","tag-equifinality","tag-least-squares","tag-nonlinear","tag-null-space","tag-observation-operator"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.8.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>The least squares method in data assimilation - DARC<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/research.reading.ac.uk\/met-darc\/2025\/04\/03\/the-least-squares-method-in-data-assimilation\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The least squares method in data assimilation - DARC\" \/>\n<meta property=\"og:description\" content=\"by Ross Bannister, April 2025 The method of least squares (or MLS, Barlow, 1993) is at the very centre of variational data assimilation (or \u201cVar\u201d, Schlatter, 2000). Var is one...Read More &gt;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/research.reading.ac.uk\/met-darc\/2025\/04\/03\/the-least-squares-method-in-data-assimilation\/\" \/>\n<meta property=\"og:site_name\" content=\"DARC\" \/>\n<meta property=\"article:published_time\" content=\"2025-04-03T09:06:53+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-13T18:55:38+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/research.reading.ac.uk\/met-darc\/wp-content\/uploads\/sites\/48\/2025\/04\/Ruggero-Boscovich-shape-least-squares-graph-Earth.gif\" \/>\n\t<meta property=\"og:image:width\" content=\"300\" \/>\n\t<meta property=\"og:image:height\" content=\"207\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/gif\" \/>\n<meta name=\"author\" content=\"Gwyneth Matthews\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Gwyneth Matthews\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/research.reading.ac.uk\/met-darc\/2025\/04\/03\/the-least-squares-method-in-data-assimilation\/\",\"url\":\"https:\/\/research.reading.ac.uk\/met-darc\/2025\/04\/03\/the-least-squares-method-in-data-assimilation\/\",\"name\":\"The least squares method in data assimilation - DARC\",\"isPartOf\":{\"@id\":\"https:\/\/research.reading.ac.uk\/met-darc\/#website\"},\"datePublished\":\"2025-04-03T09:06:53+00:00\",\"dateModified\":\"2025-04-13T18:55:38+00:00\",\"author\":{\"@id\":\"https:\/\/research.reading.ac.uk\/met-darc\/#\/schema\/person\/8e7b031a8323e2548dde20dd3a8f7e06\"},\"breadcrumb\":{\"@id\":\"https:\/\/research.reading.ac.uk\/met-darc\/2025\/04\/03\/the-least-squares-method-in-data-assimilation\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/research.reading.ac.uk\/met-darc\/2025\/04\/03\/the-least-squares-method-in-data-assimilation\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/research.reading.ac.uk\/met-darc\/2025\/04\/03\/the-least-squares-method-in-data-assimilation\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/research.reading.ac.uk\/met-darc\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"The least squares method in data assimilation\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/research.reading.ac.uk\/met-darc\/#website\",\"url\":\"https:\/\/research.reading.ac.uk\/met-darc\/\",\"name\":\"DARC\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/research.reading.ac.uk\/met-darc\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/research.reading.ac.uk\/met-darc\/#\/schema\/person\/8e7b031a8323e2548dde20dd3a8f7e06\",\"name\":\"Gwyneth Matthews\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/research.reading.ac.uk\/met-darc\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/31d02c8993ea5caae14e26e700f52ffd0696d8011a7deb1825db0f2a9a1053f0?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/31d02c8993ea5caae14e26e700f52ffd0696d8011a7deb1825db0f2a9a1053f0?s=96&d=mm&r=g\",\"caption\":\"Gwyneth Matthews\"},\"url\":\"https:\/\/research.reading.ac.uk\/met-darc\/author\/an823455\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"The least squares method in data assimilation - DARC","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/research.reading.ac.uk\/met-darc\/2025\/04\/03\/the-least-squares-method-in-data-assimilation\/","og_locale":"en_GB","og_type":"article","og_title":"The least squares method in data assimilation - DARC","og_description":"by Ross Bannister, April 2025 The method of least squares (or MLS, Barlow, 1993) is at the very centre of variational data assimilation (or \u201cVar\u201d, Schlatter, 2000). Var is one...Read More >","og_url":"https:\/\/research.reading.ac.uk\/met-darc\/2025\/04\/03\/the-least-squares-method-in-data-assimilation\/","og_site_name":"DARC","article_published_time":"2025-04-03T09:06:53+00:00","article_modified_time":"2025-04-13T18:55:38+00:00","og_image":[{"width":300,"height":207,"url":"https:\/\/research.reading.ac.uk\/met-darc\/wp-content\/uploads\/sites\/48\/2025\/04\/Ruggero-Boscovich-shape-least-squares-graph-Earth.gif","type":"image\/gif"}],"author":"Gwyneth Matthews","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Gwyneth Matthews","Estimated reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/research.reading.ac.uk\/met-darc\/2025\/04\/03\/the-least-squares-method-in-data-assimilation\/","url":"https:\/\/research.reading.ac.uk\/met-darc\/2025\/04\/03\/the-least-squares-method-in-data-assimilation\/","name":"The least squares method in data assimilation - DARC","isPartOf":{"@id":"https:\/\/research.reading.ac.uk\/met-darc\/#website"},"datePublished":"2025-04-03T09:06:53+00:00","dateModified":"2025-04-13T18:55:38+00:00","author":{"@id":"https:\/\/research.reading.ac.uk\/met-darc\/#\/schema\/person\/8e7b031a8323e2548dde20dd3a8f7e06"},"breadcrumb":{"@id":"https:\/\/research.reading.ac.uk\/met-darc\/2025\/04\/03\/the-least-squares-method-in-data-assimilation\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/research.reading.ac.uk\/met-darc\/2025\/04\/03\/the-least-squares-method-in-data-assimilation\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/research.reading.ac.uk\/met-darc\/2025\/04\/03\/the-least-squares-method-in-data-assimilation\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/research.reading.ac.uk\/met-darc\/"},{"@type":"ListItem","position":2,"name":"The least squares method in data assimilation"}]},{"@type":"WebSite","@id":"https:\/\/research.reading.ac.uk\/met-darc\/#website","url":"https:\/\/research.reading.ac.uk\/met-darc\/","name":"DARC","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/research.reading.ac.uk\/met-darc\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Person","@id":"https:\/\/research.reading.ac.uk\/met-darc\/#\/schema\/person\/8e7b031a8323e2548dde20dd3a8f7e06","name":"Gwyneth Matthews","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/research.reading.ac.uk\/met-darc\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/31d02c8993ea5caae14e26e700f52ffd0696d8011a7deb1825db0f2a9a1053f0?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/31d02c8993ea5caae14e26e700f52ffd0696d8011a7deb1825db0f2a9a1053f0?s=96&d=mm&r=g","caption":"Gwyneth Matthews"},"url":"https:\/\/research.reading.ac.uk\/met-darc\/author\/an823455\/"}]}},"cc_featured_image_caption":{"caption_text":"","source_text":"","source_url":""},"_links":{"self":[{"href":"https:\/\/research.reading.ac.uk\/met-darc\/wp-json\/wp\/v2\/posts\/2625","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/research.reading.ac.uk\/met-darc\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/research.reading.ac.uk\/met-darc\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/research.reading.ac.uk\/met-darc\/wp-json\/wp\/v2\/users\/931"}],"replies":[{"embeddable":true,"href":"https:\/\/research.reading.ac.uk\/met-darc\/wp-json\/wp\/v2\/comments?post=2625"}],"version-history":[{"count":10,"href":"https:\/\/research.reading.ac.uk\/met-darc\/wp-json\/wp\/v2\/posts\/2625\/revisions"}],"predecessor-version":[{"id":2661,"href":"https:\/\/research.reading.ac.uk\/met-darc\/wp-json\/wp\/v2\/posts\/2625\/revisions\/2661"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/research.reading.ac.uk\/met-darc\/wp-json\/wp\/v2\/media\/2649"}],"wp:attachment":[{"href":"https:\/\/research.reading.ac.uk\/met-darc\/wp-json\/wp\/v2\/media?parent=2625"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/research.reading.ac.uk\/met-darc\/wp-json\/wp\/v2\/categories?post=2625"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/research.reading.ac.uk\/met-darc\/wp-json\/wp\/v2\/tags?post=2625"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}