{"id":30002,"date":"2024-09-25T11:30:51","date_gmt":"2024-09-25T10:30:51","guid":{"rendered":"https:\/\/research.reading.ac.uk\/research-blog\/?p=30002"},"modified":"2024-09-25T11:26:33","modified_gmt":"2024-09-25T10:26:33","slug":"peer-review-and-ai-best-practice","status":"publish","type":"post","link":"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/","title":{"rendered":"Peer review and AI: what will this mean for best practice?"},"content":{"rendered":"

[vc_row][vc_column][vc_single_image image=”30007″ img_size=”full” add_caption=”yes” alignment=”center”][vc_column_text]Have you been asked to do a peer review for a journal recently? The chances are that at least one invitation may have landed in your inbox as the new academic year approaches. Did you quietly decline or did you add the request to your growing to-do list? This week is <\/em><\/strong>Peer Review Week<\/em><\/strong><\/a> and Karen Rowlett reflects on what new technologies and AI may mean for best practice and the ethics of the review process.<\/em><\/strong><\/p>\n

The number of manuscripts submitted to academic journals is soaring. Between 2016 and 2022 it has been estimated that the number of papers received per year has grown from 1.92 million to 2.82 million<\/a>. As peer reviewers are usually busy academics with their own research, teaching and administration commitments, and are not paid or recognised for their contributions, how can journal editors ever find enough peer reviewers to appraise journal submissions? New technologies such as artificial intelligence (AI) offer opportunities to streamline the peer review process but may also introduce more complex ethical problems.<\/p>\n

For instance, how many times have you been asked to review a paper that is not related to your research area? One immediate benefit of AI technology may be in more accurate targeting of requests so that the paper you are asked to review is a closer match to your expertise. This should increase reviewer uptake, speed up the review process and improve the quality of the reviews.<\/p>\n

AI tools may also be useful to pre-screen manuscripts for potential issues with research integrity such as image manipulation, plagiarism, reporting of non-existent cell lines and probes<\/a>, inappropriate or missing citations, missing consent forms or non-adherence to reporting guidelines for human clinical trials. These are all issues that can be difficult for reviewers to spot but can have a significant impact on the integrity of the scholarly record. AI may also be better than humans at detecting submissions from paper mills.[\/vc_column_text][\/vc_column][\/vc_row][vc_row][vc_column][vc_single_image image=”30008″ img_size=”full”][\/vc_column][\/vc_row][vc_row][vc_column][vc_column_text]Could AI eventually replace human peer reviewers? There\u2019s some evidence that large-language models are already being used in peer review. While using tools such as Chat GPT to refine your prose might be acceptable, particularly for non-native English speakers, relying on these tools to make decisions or write review reports could be much more problematic. A recent study of peer review reports<\/a> for conference abstracts reported that AI influence was much more likely to be detected in reports submitted close to the reviewer\u2019s deadline, suggesting that hard-pressed academics are already using it as a short cut.<\/p>\n

What kind of peer review reports might be generated by AI? Tools such as Chat GPT might be useful to generate a summary of a submitted manuscript but the whole point of peer review is that an assessment of the quality<\/em> of the research is made by an experienced r<\/em>esearcher in the field. In the study of conference abstract peer reviews, the text of the reviews appeared to be more bland and convergent when AI was used. The AI-generated reviews also seemed to be more likely\u00a0 to recommend further experiments than expert peer reviewers \u2013 a trend that might lead to more \u2018reviewer 2\u2019 angst for authors (when overly critical comments can be offensive or discouraging for authors). AI tools that produce low-quality, bland and non-committal peer reviews will surely only make the situation even worse.<\/p>\n

There are potential ethical issues with peer reviewers using AI tools to assess manuscripts without the permission of the authors. Such conduct would violate the confidentiality of peer review, go against journals\u2019 reviewer guidelines, breach copyright and place the author\u2019s work in the public domain. In order to deter such activity, publishers have already started adding guidance on the use of AI in peer review<\/a> and how to detect it in reports to their instructions to reviewers and editors.<\/p>\n

Could AI tools improve bias and prejudice in peer review decisions? Bias in peer review and editorial decisions has been well documented and is known to disadvantage authors based on their gender, ethnicity, age, experience, race and geographical location. Switching to AI-based decisions might not necessarily result in a fairer outcome. Unfortunately, AI tools can also have inherent biases depending on the data that they are trained on, particularly if it is from historical data, and depending on who developed the tool and how the software is designed.<\/p>\n

Despite all these reservations, it seems likely that AI-based tools will be integrated into the peer review process by publishers in the near future. Key elements to consider will be transparency and accountability. Authors, reviewers and readers will need to know how AI was used in the submission process \u2013 and it should never be a direct replacement for human judgement. It must be clear to all involved which parts of journal workflow are powered by AI decisions. Authors and reviewers should also be able to challenge decisions made on the basis of AI in the same way that they can currently dispute other editorial decisions. Well -developed AI tools used responsibly at the right point in the journal submission process do have the potential to streamline peer review and make the whole ordeal easier for reviewers and authors alike.<\/p>\n

 <\/p>\n

Karen Rowlett is Research Publications Adviser at the University of Reading.[\/vc_column_text][\/vc_column][\/vc_row][vc_row][vc_column][\/vc_column][\/vc_row]<\/p>\n","protected":false},"excerpt":{"rendered":"

[vc_row][vc_column][vc_single_image image=”30007″ img_size=”full” add_caption=”yes” alignment=”center”][vc_column_text]Have you been asked to do a peer review for a journal recently? The chances are that at least one invitation may have landed in your…Read More ><\/a><\/p>\n","protected":false},"author":969,"featured_media":30007,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"__cvm_playback_settings":[],"__cvm_video_id":"","footnotes":""},"categories":[1],"tags":[49,81,2717,2716,755],"class_list":["post-30002","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorised","tag-ai","tag-artificial-intelligence","tag-peer-review","tag-professional-services","tag-research"],"acf":[],"yoast_head":"\nPeer review and AI: what will this mean for best practice? - Connecting Research<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Peer review and AI: what will this mean for best practice? - Connecting Research\" \/>\n<meta property=\"og:description\" content=\"[vc_row][vc_column][vc_single_image image=”30007″ img_size=”full” add_caption=”yes” alignment=”center”][vc_column_text]Have you been asked to do a peer review for a journal recently? The chances are that at least one invitation may have landed in your...Read More >\" \/>\n<meta property=\"og:url\" content=\"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/\" \/>\n<meta property=\"og:site_name\" content=\"Connecting Research\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/theuniversityofreading\/\" \/>\n<meta property=\"article:published_time\" content=\"2024-09-25T10:30:51+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-09-25T10:26:33+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/research.reading.ac.uk\/research-blog\/wp-content\/uploads\/sites\/72\/2024\/09\/audit-4576720_1280.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1280\" \/>\n\t<meta property=\"og:image:height\" content=\"825\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Conor Byrne\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@UniRdg_Research\" \/>\n<meta name=\"twitter:site\" content=\"@UniRdg_Research\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Conor Byrne\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/\"},\"author\":{\"name\":\"Conor Byrne\",\"@id\":\"https:\/\/research.reading.ac.uk\/research-blog\/#\/schema\/person\/0f3f2225d484dd8910ebd7758671acbf\"},\"headline\":\"Peer review and AI: what will this mean for best practice?\",\"datePublished\":\"2024-09-25T10:30:51+00:00\",\"dateModified\":\"2024-09-25T10:26:33+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/\"},\"wordCount\":940,\"publisher\":{\"@id\":\"https:\/\/research.reading.ac.uk\/research-blog\/#organization\"},\"keywords\":[\"AI\",\"artificial intelligence\",\"peer review\",\"Professional services\",\"research\"],\"inLanguage\":\"en-GB\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/\",\"url\":\"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/\",\"name\":\"Peer review and AI: what will this mean for best practice? - Connecting Research\",\"isPartOf\":{\"@id\":\"https:\/\/research.reading.ac.uk\/research-blog\/#website\"},\"datePublished\":\"2024-09-25T10:30:51+00:00\",\"dateModified\":\"2024-09-25T10:26:33+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/research.reading.ac.uk\/research-blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Peer review and AI: what will this mean for best practice?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/research.reading.ac.uk\/research-blog\/#website\",\"url\":\"https:\/\/research.reading.ac.uk\/research-blog\/\",\"name\":\"Connecting Research\",\"description\":\"Research stories from across the University of Reading\",\"publisher\":{\"@id\":\"https:\/\/research.reading.ac.uk\/research-blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/research.reading.ac.uk\/research-blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/research.reading.ac.uk\/research-blog\/#organization\",\"name\":\"University of Reading\",\"alternateName\":\"UoR\",\"url\":\"https:\/\/research.reading.ac.uk\/research-blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/research.reading.ac.uk\/research-blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/research.reading.ac.uk\/research-blog\/wp-content\/uploads\/sites\/72\/2022\/10\/University-of-Reading-Logo-1.png\",\"contentUrl\":\"https:\/\/research.reading.ac.uk\/research-blog\/wp-content\/uploads\/sites\/72\/2022\/10\/University-of-Reading-Logo-1.png\",\"width\":440,\"height\":147,\"caption\":\"University of Reading\"},\"image\":{\"@id\":\"https:\/\/research.reading.ac.uk\/research-blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/theuniversityofreading\/\",\"https:\/\/twitter.com\/UniRdg_Research\",\"https:\/\/www.linkedin.com\/school\/university-of-reading\/mycompany\/verification\/\",\"https:\/\/www.youtube.com\/channel\/UCsFUmxmcVbVJ-9iGczsKfuw\",\"https:\/\/en.wikipedia.org\/wiki\/University_of_Reading\",\"https:\/\/www.instagram.com\/uniofreading\/?hl=en\",\"https:\/\/www.tiktok.com\/@uniofreading\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/research.reading.ac.uk\/research-blog\/#\/schema\/person\/0f3f2225d484dd8910ebd7758671acbf\",\"name\":\"Conor Byrne\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/research.reading.ac.uk\/research-blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/a60519e3600bcaf0362aafdb23be4ec4315094a43eaa460a2f681845f306ea58?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/a60519e3600bcaf0362aafdb23be4ec4315094a43eaa460a2f681845f306ea58?s=96&d=mm&r=g\",\"caption\":\"Conor Byrne\"},\"url\":\"https:\/\/research.reading.ac.uk\/research-blog\/author\/ci931957\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Peer review and AI: what will this mean for best practice? - Connecting Research","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/","og_locale":"en_GB","og_type":"article","og_title":"Peer review and AI: what will this mean for best practice? - Connecting Research","og_description":"[vc_row][vc_column][vc_single_image image=”30007″ img_size=”full” add_caption=”yes” alignment=”center”][vc_column_text]Have you been asked to do a peer review for a journal recently? The chances are that at least one invitation may have landed in your...Read More >","og_url":"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/","og_site_name":"Connecting Research","article_publisher":"https:\/\/www.facebook.com\/theuniversityofreading\/","article_published_time":"2024-09-25T10:30:51+00:00","article_modified_time":"2024-09-25T10:26:33+00:00","og_image":[{"width":1280,"height":825,"url":"https:\/\/research.reading.ac.uk\/research-blog\/wp-content\/uploads\/sites\/72\/2024\/09\/audit-4576720_1280.jpg","type":"image\/jpeg"}],"author":"Conor Byrne","twitter_card":"summary_large_image","twitter_creator":"@UniRdg_Research","twitter_site":"@UniRdg_Research","twitter_misc":{"Written by":"Conor Byrne","Estimated reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/#article","isPartOf":{"@id":"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/"},"author":{"name":"Conor Byrne","@id":"https:\/\/research.reading.ac.uk\/research-blog\/#\/schema\/person\/0f3f2225d484dd8910ebd7758671acbf"},"headline":"Peer review and AI: what will this mean for best practice?","datePublished":"2024-09-25T10:30:51+00:00","dateModified":"2024-09-25T10:26:33+00:00","mainEntityOfPage":{"@id":"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/"},"wordCount":940,"publisher":{"@id":"https:\/\/research.reading.ac.uk\/research-blog\/#organization"},"keywords":["AI","artificial intelligence","peer review","Professional services","research"],"inLanguage":"en-GB"},{"@type":"WebPage","@id":"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/","url":"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/","name":"Peer review and AI: what will this mean for best practice? - Connecting Research","isPartOf":{"@id":"https:\/\/research.reading.ac.uk\/research-blog\/#website"},"datePublished":"2024-09-25T10:30:51+00:00","dateModified":"2024-09-25T10:26:33+00:00","breadcrumb":{"@id":"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/research.reading.ac.uk\/research-blog\/2024\/09\/25\/peer-review-and-ai-best-practice\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/research.reading.ac.uk\/research-blog\/"},{"@type":"ListItem","position":2,"name":"Peer review and AI: what will this mean for best practice?"}]},{"@type":"WebSite","@id":"https:\/\/research.reading.ac.uk\/research-blog\/#website","url":"https:\/\/research.reading.ac.uk\/research-blog\/","name":"Connecting Research","description":"Research stories from across the University of Reading","publisher":{"@id":"https:\/\/research.reading.ac.uk\/research-blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/research.reading.ac.uk\/research-blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/research.reading.ac.uk\/research-blog\/#organization","name":"University of Reading","alternateName":"UoR","url":"https:\/\/research.reading.ac.uk\/research-blog\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/research.reading.ac.uk\/research-blog\/#\/schema\/logo\/image\/","url":"https:\/\/research.reading.ac.uk\/research-blog\/wp-content\/uploads\/sites\/72\/2022\/10\/University-of-Reading-Logo-1.png","contentUrl":"https:\/\/research.reading.ac.uk\/research-blog\/wp-content\/uploads\/sites\/72\/2022\/10\/University-of-Reading-Logo-1.png","width":440,"height":147,"caption":"University of Reading"},"image":{"@id":"https:\/\/research.reading.ac.uk\/research-blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/theuniversityofreading\/","https:\/\/twitter.com\/UniRdg_Research","https:\/\/www.linkedin.com\/school\/university-of-reading\/mycompany\/verification\/","https:\/\/www.youtube.com\/channel\/UCsFUmxmcVbVJ-9iGczsKfuw","https:\/\/en.wikipedia.org\/wiki\/University_of_Reading","https:\/\/www.instagram.com\/uniofreading\/?hl=en","https:\/\/www.tiktok.com\/@uniofreading"]},{"@type":"Person","@id":"https:\/\/research.reading.ac.uk\/research-blog\/#\/schema\/person\/0f3f2225d484dd8910ebd7758671acbf","name":"Conor Byrne","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/research.reading.ac.uk\/research-blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/a60519e3600bcaf0362aafdb23be4ec4315094a43eaa460a2f681845f306ea58?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/a60519e3600bcaf0362aafdb23be4ec4315094a43eaa460a2f681845f306ea58?s=96&d=mm&r=g","caption":"Conor Byrne"},"url":"https:\/\/research.reading.ac.uk\/research-blog\/author\/ci931957\/"}]}},"cc_featured_image_caption":{"caption_text":"","source_text":"","source_url":""},"_links":{"self":[{"href":"https:\/\/research.reading.ac.uk\/research-blog\/wp-json\/wp\/v2\/posts\/30002","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/research.reading.ac.uk\/research-blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/research.reading.ac.uk\/research-blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/research.reading.ac.uk\/research-blog\/wp-json\/wp\/v2\/users\/969"}],"replies":[{"embeddable":true,"href":"https:\/\/research.reading.ac.uk\/research-blog\/wp-json\/wp\/v2\/comments?post=30002"}],"version-history":[{"count":7,"href":"https:\/\/research.reading.ac.uk\/research-blog\/wp-json\/wp\/v2\/posts\/30002\/revisions"}],"predecessor-version":[{"id":30011,"href":"https:\/\/research.reading.ac.uk\/research-blog\/wp-json\/wp\/v2\/posts\/30002\/revisions\/30011"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/research.reading.ac.uk\/research-blog\/wp-json\/wp\/v2\/media\/30007"}],"wp:attachment":[{"href":"https:\/\/research.reading.ac.uk\/research-blog\/wp-json\/wp\/v2\/media?parent=30002"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/research.reading.ac.uk\/research-blog\/wp-json\/wp\/v2\/categories?post=30002"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/research.reading.ac.uk\/research-blog\/wp-json\/wp\/v2\/tags?post=30002"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}