{"id":773,"date":"2020-07-20T08:58:30","date_gmt":"2020-07-20T07:58:30","guid":{"rendered":"https:\/\/research.reading.ac.uk\/palaeoclimate\/?p=773"},"modified":"2021-01-29T14:43:55","modified_gmt":"2021-01-29T14:43:55","slug":"benchmarking-is-a-necessary-evil","status":"publish","type":"post","link":"https:\/\/research.reading.ac.uk\/palaeoclimate\/benchmarking-is-a-necessary-evil\/","title":{"rendered":"Benchmarking is a necessary evil. By Sandy Harrison"},"content":{"rendered":"

The FireMIP paper describing the benchmarking of fire-enabled vegetation model simulations of the historic period is now out in Geoscientific Model Development (https:\/\/doi.org\/10.5194\/gmd-13-3299-2020<\/a>). The team, led by Stijn Hantson, have done a thorough job of evaluating the nine models that contributed simulations to FireMIP against a variety of vegetation- and fire-related benchmark data sets. The conclusions, however, are dispiriting. With the exception of two relatively “old” models, LPJ\u2013GUESS\u2013 GlobFIRM and MC2 which clearly perform less well, all of the other models bunch together in the ratings. None of these new models clearly outperforms the others, and furthermore most of them perform less well than a null model created from the mean of the observations for variables such as the seasonal concentration (i.e. the length of the fire season) and the interannual variability in burnt area. Perhaps unsurprisingly, most of the models simulate vegetation parameters (such as global production, leaf area index, and vegetation carbon storage) better than they simulated fire-related variables. Several other FireMIP papers have looked at specific aspects of these simulations — and the bottom line is that we have a long way to go before we can expect to produce realistic simulations of fire regimes or reliable predictions of potential future changes. Members of the Leverhulme Centre for Wildfires, Environment and Society (https:\/\/centreforwildfires.org\/<\/a>) have been involved in FireMIP since its inauguration, and perhaps the FireMIP exercise is one of the things that is spurring us to find creative new ways to analyse and model fire. Hopefully, the work of members of SPECIAL, in particularly on the role of fuel accumulation (Alex) and on fire-vegetation and climate-fire relationships in the past (David, Yicheng) will be helpful here. Benchmarking is a necessary evil, but we definitely need new insights into the complex interactions between climate-vegetation-fire and human activities in order to be able to create better models.<\/p>\n

\"\"<\/p>\n

The range in interannual variability in burnt area for the years 2001\u20132012 for all models and burnt area datasets which span the entire time period (GFED4, GFED4s, MCD45, FireCCI51). Results from the individual FireMIP models, as well as the observational minimum-maximum values, are plotted.<\/h5>\n","protected":false},"excerpt":{"rendered":"

The FireMIP paper describing the benchmarking of fire-enabled vegetation model simulations of the historic period is now out in Geoscientific Model Development (https:\/\/doi.org\/10.5194\/gmd-13-3299-2020). The team, led by Stijn Hantson, have…Read More ><\/a><\/p>\n","protected":false},"author":228,"featured_media":775,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"__cvm_playback_settings":[],"__cvm_video_id":"","footnotes":""},"categories":[22],"tags":[50,10,12,30,14,2,53,41],"class_list":["post-773","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog","tag-benchmarking","tag-fire","tag-firemip","tag-model-evaluation","tag-partner-institutions","tag-publication","tag-vegetation","tag-wildfire"],"acf":[],"yoast_head":"\nBenchmarking is a necessary evil. By Sandy Harrison - SPECIAL Palaeoclimate<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/research.reading.ac.uk\/palaeoclimate\/benchmarking-is-a-necessary-evil\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Benchmarking is a necessary evil. By Sandy Harrison - SPECIAL Palaeoclimate\" \/>\n<meta property=\"og:description\" content=\"The FireMIP paper describing the benchmarking of fire-enabled vegetation model simulations of the historic period is now out in Geoscientific Model Development (https:\/\/doi.org\/10.5194\/gmd-13-3299-2020). The team, led by Stijn Hantson, have...Read More >\" \/>\n<meta property=\"og:url\" content=\"https:\/\/research.reading.ac.uk\/palaeoclimate\/benchmarking-is-a-necessary-evil\/\" \/>\n<meta property=\"og:site_name\" content=\"SPECIAL Palaeoclimate\" \/>\n<meta property=\"article:published_time\" content=\"2020-07-20T07:58:30+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-01-29T14:43:55+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/research.reading.ac.uk\/palaeoclimate\/wp-content\/uploads\/sites\/78\/Unorganized\/gmd-13-3299-2020-f05-web-e1595231733251.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1074\" \/>\n\t<meta property=\"og:image:height\" content=\"296\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Laia Comas Bru\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Laia Comas Bru\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/research.reading.ac.uk\/palaeoclimate\/benchmarking-is-a-necessary-evil\/\",\"url\":\"https:\/\/research.reading.ac.uk\/palaeoclimate\/benchmarking-is-a-necessary-evil\/\",\"name\":\"Benchmarking is a necessary evil. By Sandy Harrison - SPECIAL Palaeoclimate\",\"isPartOf\":{\"@id\":\"https:\/\/research.reading.ac.uk\/palaeoclimate\/#website\"},\"datePublished\":\"2020-07-20T07:58:30+00:00\",\"dateModified\":\"2021-01-29T14:43:55+00:00\",\"author\":{\"@id\":\"https:\/\/research.reading.ac.uk\/palaeoclimate\/#\/schema\/person\/777650b37a6a92704ee68662ff50e9d8\"},\"breadcrumb\":{\"@id\":\"https:\/\/research.reading.ac.uk\/palaeoclimate\/benchmarking-is-a-necessary-evil\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/research.reading.ac.uk\/palaeoclimate\/benchmarking-is-a-necessary-evil\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/research.reading.ac.uk\/palaeoclimate\/benchmarking-is-a-necessary-evil\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/research.reading.ac.uk\/palaeoclimate\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Benchmarking is a necessary evil. By Sandy Harrison\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/research.reading.ac.uk\/palaeoclimate\/#website\",\"url\":\"https:\/\/research.reading.ac.uk\/palaeoclimate\/\",\"name\":\"SPECIAL Palaeoclimate\",\"description\":\"Webpage of Sandy's PalaeoEnvironments and Climate Analysis research group at the University of Reading (UK)\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/research.reading.ac.uk\/palaeoclimate\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/research.reading.ac.uk\/palaeoclimate\/#\/schema\/person\/777650b37a6a92704ee68662ff50e9d8\",\"name\":\"Laia Comas Bru\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/research.reading.ac.uk\/palaeoclimate\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/4b2b905206c1258c10bf434e14bee50c76be5ee6b340b27edc8b882a7568d17c?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/4b2b905206c1258c10bf434e14bee50c76be5ee6b340b27edc8b882a7568d17c?s=96&d=mm&r=g\",\"caption\":\"Laia Comas Bru\"},\"url\":\"https:\/\/research.reading.ac.uk\/palaeoclimate\/author\/l-comasbrureading-ac-uk\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Benchmarking is a necessary evil. By Sandy Harrison - SPECIAL Palaeoclimate","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/research.reading.ac.uk\/palaeoclimate\/benchmarking-is-a-necessary-evil\/","og_locale":"en_GB","og_type":"article","og_title":"Benchmarking is a necessary evil. By Sandy Harrison - SPECIAL Palaeoclimate","og_description":"The FireMIP paper describing the benchmarking of fire-enabled vegetation model simulations of the historic period is now out in Geoscientific Model Development (https:\/\/doi.org\/10.5194\/gmd-13-3299-2020). The team, led by Stijn Hantson, have...Read More >","og_url":"https:\/\/research.reading.ac.uk\/palaeoclimate\/benchmarking-is-a-necessary-evil\/","og_site_name":"SPECIAL Palaeoclimate","article_published_time":"2020-07-20T07:58:30+00:00","article_modified_time":"2021-01-29T14:43:55+00:00","og_image":[{"width":1074,"height":296,"url":"https:\/\/research.reading.ac.uk\/palaeoclimate\/wp-content\/uploads\/sites\/78\/Unorganized\/gmd-13-3299-2020-f05-web-e1595231733251.png","type":"image\/png"}],"author":"Laia Comas Bru","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Laia Comas Bru","Estimated reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/research.reading.ac.uk\/palaeoclimate\/benchmarking-is-a-necessary-evil\/","url":"https:\/\/research.reading.ac.uk\/palaeoclimate\/benchmarking-is-a-necessary-evil\/","name":"Benchmarking is a necessary evil. By Sandy Harrison - SPECIAL Palaeoclimate","isPartOf":{"@id":"https:\/\/research.reading.ac.uk\/palaeoclimate\/#website"},"datePublished":"2020-07-20T07:58:30+00:00","dateModified":"2021-01-29T14:43:55+00:00","author":{"@id":"https:\/\/research.reading.ac.uk\/palaeoclimate\/#\/schema\/person\/777650b37a6a92704ee68662ff50e9d8"},"breadcrumb":{"@id":"https:\/\/research.reading.ac.uk\/palaeoclimate\/benchmarking-is-a-necessary-evil\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/research.reading.ac.uk\/palaeoclimate\/benchmarking-is-a-necessary-evil\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/research.reading.ac.uk\/palaeoclimate\/benchmarking-is-a-necessary-evil\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/research.reading.ac.uk\/palaeoclimate\/"},{"@type":"ListItem","position":2,"name":"Benchmarking is a necessary evil. By Sandy Harrison"}]},{"@type":"WebSite","@id":"https:\/\/research.reading.ac.uk\/palaeoclimate\/#website","url":"https:\/\/research.reading.ac.uk\/palaeoclimate\/","name":"SPECIAL Palaeoclimate","description":"Webpage of Sandy's PalaeoEnvironments and Climate Analysis research group at the University of Reading (UK)","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/research.reading.ac.uk\/palaeoclimate\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Person","@id":"https:\/\/research.reading.ac.uk\/palaeoclimate\/#\/schema\/person\/777650b37a6a92704ee68662ff50e9d8","name":"Laia Comas Bru","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/research.reading.ac.uk\/palaeoclimate\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/4b2b905206c1258c10bf434e14bee50c76be5ee6b340b27edc8b882a7568d17c?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/4b2b905206c1258c10bf434e14bee50c76be5ee6b340b27edc8b882a7568d17c?s=96&d=mm&r=g","caption":"Laia Comas Bru"},"url":"https:\/\/research.reading.ac.uk\/palaeoclimate\/author\/l-comasbrureading-ac-uk\/"}]}},"cc_featured_image_caption":{"caption_text":false,"source_text":false,"source_url":false},"_links":{"self":[{"href":"https:\/\/research.reading.ac.uk\/palaeoclimate\/wp-json\/wp\/v2\/posts\/773","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/research.reading.ac.uk\/palaeoclimate\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/research.reading.ac.uk\/palaeoclimate\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/research.reading.ac.uk\/palaeoclimate\/wp-json\/wp\/v2\/users\/228"}],"replies":[{"embeddable":true,"href":"https:\/\/research.reading.ac.uk\/palaeoclimate\/wp-json\/wp\/v2\/comments?post=773"}],"version-history":[{"count":2,"href":"https:\/\/research.reading.ac.uk\/palaeoclimate\/wp-json\/wp\/v2\/posts\/773\/revisions"}],"predecessor-version":[{"id":777,"href":"https:\/\/research.reading.ac.uk\/palaeoclimate\/wp-json\/wp\/v2\/posts\/773\/revisions\/777"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/research.reading.ac.uk\/palaeoclimate\/wp-json\/wp\/v2\/media\/775"}],"wp:attachment":[{"href":"https:\/\/research.reading.ac.uk\/palaeoclimate\/wp-json\/wp\/v2\/media?parent=773"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/research.reading.ac.uk\/palaeoclimate\/wp-json\/wp\/v2\/categories?post=773"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/research.reading.ac.uk\/palaeoclimate\/wp-json\/wp\/v2\/tags?post=773"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}