{"id":542,"date":"2016-12-19T11:55:46","date_gmt":"2016-12-19T11:55:46","guid":{"rendered":"http:\/\/blogs.reading.ac.uk\/flooding\/?p=542"},"modified":"2019-10-07T16:00:49","modified_gmt":"2019-10-07T15:00:49","slug":"the-challenges-of-using-big-data-in-numerical-weather-prediction-meteorological-observations-from-air-traffic-control-reports","status":"publish","type":"post","link":"https:\/\/research.reading.ac.uk\/flooding\/the-challenges-of-using-big-data-in-numerical-weather-prediction-meteorological-observations-from-air-traffic-control-reports\/","title":{"rendered":"The challenges of using \u201cbig data\u201d in Numerical Weather Prediction: meteorological observations from air traffic control reports."},"content":{"rendered":"\n\n\n
\"sarah\"<\/a><\/td>\nBy Dr. Sarah Dance (University of Reading)<\/em>
\n19th December 20146<\/em><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

\u00a0<\/span>Many would say that Numerical Weather Prediction has been using \u201cbig data\u201d for decades. Routine forecasts are produced using computational models with billions of variables, and tens of millions of observations, several times a day. Most of these observations come from scientifically designed observing networks, such as satellite instruments, weather radar and carefully sited weather stations. However, urban areas also present rich sources of data, that to date have not been fully explored or exploited (e.g., citizen science, smartphones, internet of things etc.), and could provide significant benefits when forecasting on small scales, at low cost.<\/p>\n

In surface scientific networks, point observations are often sited away from buildings, in locations that are intended to be more broadly representative of larger areas and not designed to reflect local urban conditions. These observations lend themselves more naturally to comparison with discretized models, whose grid-lengths may be much larger than the size of a building. For datasets of opportunity, a key problem is to understand the effects of the urban environment on the observations so that uncertainties can be properly attributed and proper quality control procedures established. Furthermore there are complex issues surrounding use of the data, such as personal privacy and data ownership that must be overcome.<\/p>\n

 <\/p>\n

\"planes\"<\/p>\n

For the rest of this article we focus one dataset of opportunity arising from air traffic control radar reports. Mode Selective Enhanced Surveillance (Mode-S EHS) is used by Air Traffic Management to retrieve routine reports on an aircraft’s state vector at a high temporal frequency (every 4 to 12 seconds). The state vector consists of true airspeed, magnetic-heading, ground-speed, ground-heading, altitude and Mach number. Mode-S EHS reports can be used to derive estimates of the ambient air temperature and horizontal wind at the aircraft’s location. These derived observations have the potential to give weather information on fine spatial and temporal scales, especially in the vicinity of airports, where there are millions of reports per day. For example high-frequency reporting of vertical profiles of temperature and wind may provide extra information for use in numerical weather prediction that would have particular value in the forecasting of hazardous weather. \u00a0While some of the problems of understanding and using datasets of opportunity are circumvented (the effects of buildings are less relevant to flying aircraft), all measurements during aircraft turns and other manoeuvres have to be discarded. Furthermore, the reports are transmitted in small data packets, with limited precision, with the result that the uncertainty in the derived meteorological observations is very large, particularly at lower altitudes. For more information see<\/p>\n

Mirza, A. K., Ballard, S. P., Dance, S. L., Maisey, P., Rooney, G. G. and Stone, E. K. (2016), Comparison of aircraft-derived observations with in situ research aircraft measurements. Q.J.R. Meteorol. Soc., 142: 2949\u20132967. doi:10.1002\/qj.2864<\/a><\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

By Dr. Sarah Dance (University of Reading) 19th December 20146 \u00a0Many would say that Numerical Weather Prediction has been using \u201cbig data\u201d for decades. Routine forecasts are produced using computational…Read More ><\/a><\/p>\n","protected":false},"author":219,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"__cvm_playback_settings":[],"__cvm_video_id":"","footnotes":""},"categories":[10,1],"tags":[],"coauthors":[],"class_list":["post-542","post","type-post","status-publish","format-standard","hentry","category-blog","category-uncategorised"],"acf":[],"yoast_head":"\nThe challenges of using \u201cbig data\u201d in Numerical Weather Prediction: meteorological observations from air traffic control reports. - Flooding from Intense Rainfall<\/title>\n<meta name=\"robots\" content=\"noindex, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The challenges of using \u201cbig data\u201d in Numerical Weather Prediction: meteorological observations from air traffic control reports. - Flooding from Intense Rainfall\" \/>\n<meta property=\"og:description\" content=\"By Dr. Sarah Dance (University of Reading) 19th December 20146 \u00a0Many would say that Numerical Weather Prediction has been using \u201cbig data\u201d for decades. Routine forecasts are produced using computational...Read More >\" \/>\n<meta property=\"og:url\" content=\"https:\/\/research.reading.ac.uk\/flooding\/the-challenges-of-using-big-data-in-numerical-weather-prediction-meteorological-observations-from-air-traffic-control-reports\/\" \/>\n<meta property=\"og:site_name\" content=\"Flooding from Intense Rainfall\" \/>\n<meta property=\"article:published_time\" content=\"2016-12-19T11:55:46+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2019-10-07T15:00:49+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/research.reading.ac.uk\/flooding\/wp-content\/uploads\/sites\/139\/2014\/08\/sarah.jpg\" \/>\n<meta name=\"author\" content=\"Alex Daykin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Alex Daykin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/research.reading.ac.uk\/flooding\/the-challenges-of-using-big-data-in-numerical-weather-prediction-meteorological-observations-from-air-traffic-control-reports\/\",\"url\":\"https:\/\/research.reading.ac.uk\/flooding\/the-challenges-of-using-big-data-in-numerical-weather-prediction-meteorological-observations-from-air-traffic-control-reports\/\",\"name\":\"The challenges of using \u201cbig data\u201d in Numerical Weather Prediction: meteorological observations from air traffic control reports. - Flooding from Intense Rainfall\",\"isPartOf\":{\"@id\":\"https:\/\/research.reading.ac.uk\/flooding\/#website\"},\"datePublished\":\"2016-12-19T11:55:46+00:00\",\"dateModified\":\"2019-10-07T15:00:49+00:00\",\"author\":{\"@id\":\"https:\/\/research.reading.ac.uk\/flooding\/#\/schema\/person\/532e1ce773445fb76d77785ea76f5be8\"},\"breadcrumb\":{\"@id\":\"https:\/\/research.reading.ac.uk\/flooding\/the-challenges-of-using-big-data-in-numerical-weather-prediction-meteorological-observations-from-air-traffic-control-reports\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/research.reading.ac.uk\/flooding\/the-challenges-of-using-big-data-in-numerical-weather-prediction-meteorological-observations-from-air-traffic-control-reports\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/research.reading.ac.uk\/flooding\/the-challenges-of-using-big-data-in-numerical-weather-prediction-meteorological-observations-from-air-traffic-control-reports\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/research.reading.ac.uk\/flooding\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"The challenges of using \u201cbig data\u201d in Numerical Weather Prediction: meteorological observations from air traffic control reports.\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/research.reading.ac.uk\/flooding\/#website\",\"url\":\"https:\/\/research.reading.ac.uk\/flooding\/\",\"name\":\"Flooding from Intense Rainfall\",\"description\":\"Project FRANC & Project SINATRA\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/research.reading.ac.uk\/flooding\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/research.reading.ac.uk\/flooding\/#\/schema\/person\/532e1ce773445fb76d77785ea76f5be8\",\"name\":\"Alex Daykin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/research.reading.ac.uk\/flooding\/#\/schema\/person\/image\/26159086a629d325cff50dd27004d579\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/48d6c4db18e4195997d9ec88cad612357d655264b37576280d9aceae3fa7aac4?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/48d6c4db18e4195997d9ec88cad612357d655264b37576280d9aceae3fa7aac4?s=96&d=mm&r=g\",\"caption\":\"Alex Daykin\"},\"url\":\"https:\/\/research.reading.ac.uk\/flooding\/author\/a-j-daykinreading-ac-uk\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"The challenges of using \u201cbig data\u201d in Numerical Weather Prediction: meteorological observations from air traffic control reports. - Flooding from Intense Rainfall","robots":{"index":"noindex","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"og_locale":"en_GB","og_type":"article","og_title":"The challenges of using \u201cbig data\u201d in Numerical Weather Prediction: meteorological observations from air traffic control reports. - Flooding from Intense Rainfall","og_description":"By Dr. Sarah Dance (University of Reading) 19th December 20146 \u00a0Many would say that Numerical Weather Prediction has been using \u201cbig data\u201d for decades. Routine forecasts are produced using computational...Read More >","og_url":"https:\/\/research.reading.ac.uk\/flooding\/the-challenges-of-using-big-data-in-numerical-weather-prediction-meteorological-observations-from-air-traffic-control-reports\/","og_site_name":"Flooding from Intense Rainfall","article_published_time":"2016-12-19T11:55:46+00:00","article_modified_time":"2019-10-07T15:00:49+00:00","og_image":[{"url":"https:\/\/research.reading.ac.uk\/flooding\/wp-content\/uploads\/sites\/139\/2014\/08\/sarah.jpg"}],"author":"Alex Daykin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Alex Daykin","Estimated reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/research.reading.ac.uk\/flooding\/the-challenges-of-using-big-data-in-numerical-weather-prediction-meteorological-observations-from-air-traffic-control-reports\/","url":"https:\/\/research.reading.ac.uk\/flooding\/the-challenges-of-using-big-data-in-numerical-weather-prediction-meteorological-observations-from-air-traffic-control-reports\/","name":"The challenges of using \u201cbig data\u201d in Numerical Weather Prediction: meteorological observations from air traffic control reports. - Flooding from Intense Rainfall","isPartOf":{"@id":"https:\/\/research.reading.ac.uk\/flooding\/#website"},"datePublished":"2016-12-19T11:55:46+00:00","dateModified":"2019-10-07T15:00:49+00:00","author":{"@id":"https:\/\/research.reading.ac.uk\/flooding\/#\/schema\/person\/532e1ce773445fb76d77785ea76f5be8"},"breadcrumb":{"@id":"https:\/\/research.reading.ac.uk\/flooding\/the-challenges-of-using-big-data-in-numerical-weather-prediction-meteorological-observations-from-air-traffic-control-reports\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/research.reading.ac.uk\/flooding\/the-challenges-of-using-big-data-in-numerical-weather-prediction-meteorological-observations-from-air-traffic-control-reports\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/research.reading.ac.uk\/flooding\/the-challenges-of-using-big-data-in-numerical-weather-prediction-meteorological-observations-from-air-traffic-control-reports\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/research.reading.ac.uk\/flooding\/"},{"@type":"ListItem","position":2,"name":"The challenges of using \u201cbig data\u201d in Numerical Weather Prediction: meteorological observations from air traffic control reports."}]},{"@type":"WebSite","@id":"https:\/\/research.reading.ac.uk\/flooding\/#website","url":"https:\/\/research.reading.ac.uk\/flooding\/","name":"Flooding from Intense Rainfall","description":"Project FRANC & Project SINATRA","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/research.reading.ac.uk\/flooding\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Person","@id":"https:\/\/research.reading.ac.uk\/flooding\/#\/schema\/person\/532e1ce773445fb76d77785ea76f5be8","name":"Alex Daykin","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/research.reading.ac.uk\/flooding\/#\/schema\/person\/image\/26159086a629d325cff50dd27004d579","url":"https:\/\/secure.gravatar.com\/avatar\/48d6c4db18e4195997d9ec88cad612357d655264b37576280d9aceae3fa7aac4?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/48d6c4db18e4195997d9ec88cad612357d655264b37576280d9aceae3fa7aac4?s=96&d=mm&r=g","caption":"Alex Daykin"},"url":"https:\/\/research.reading.ac.uk\/flooding\/author\/a-j-daykinreading-ac-uk\/"}]}},"cc_featured_image_caption":{"caption_text":false,"source_text":false,"source_url":false},"_links":{"self":[{"href":"https:\/\/research.reading.ac.uk\/flooding\/wp-json\/wp\/v2\/posts\/542","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/research.reading.ac.uk\/flooding\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/research.reading.ac.uk\/flooding\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/research.reading.ac.uk\/flooding\/wp-json\/wp\/v2\/users\/219"}],"replies":[{"embeddable":true,"href":"https:\/\/research.reading.ac.uk\/flooding\/wp-json\/wp\/v2\/comments?post=542"}],"version-history":[{"count":1,"href":"https:\/\/research.reading.ac.uk\/flooding\/wp-json\/wp\/v2\/posts\/542\/revisions"}],"predecessor-version":[{"id":658,"href":"https:\/\/research.reading.ac.uk\/flooding\/wp-json\/wp\/v2\/posts\/542\/revisions\/658"}],"wp:attachment":[{"href":"https:\/\/research.reading.ac.uk\/flooding\/wp-json\/wp\/v2\/media?parent=542"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/research.reading.ac.uk\/flooding\/wp-json\/wp\/v2\/categories?post=542"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/research.reading.ac.uk\/flooding\/wp-json\/wp\/v2\/tags?post=542"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/research.reading.ac.uk\/flooding\/wp-json\/wp\/v2\/coauthors?post=542"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}