{"id":5123,"date":"2019-10-02T19:42:39","date_gmt":"2019-10-02T18:42:39","guid":{"rendered":"https:\/\/research.reading.ac.uk\/act\/?post_type=kbe_knowledgebase&p=5123"},"modified":"2020-04-23T13:53:03","modified_gmt":"2020-04-23T12:53:03","slug":"poplar-replacement","status":"publish","type":"kbe_knowledgebase","link":"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/poplar-replacement\/","title":{"rendered":"Poplar replacement"},"content":{"rendered":"

This article is addressed to the users already using poplar for automated data transfers. It might be also useful to some users who will need to find an alternative for their automated data transfers currently running on oak.
\n<\/strong><\/p>\n

Background on poplar.reading<\/h3>\n

Poplar.reading.ac.uk<\/strong> is a server located outside the University trusted network. It is set up to accept ssh connections authenticated with ssh key, potentially without passphrase, to allow automated data transfers with rsync. Transferring data is a two step process. Remote systems are depositing data on poplar, and then the data is moved from poplar to UoR data storage by connecting to poplar from the trusted network side. The old poplar.reading.ac.uk will be permanently switched off on 19 Dec 2019<\/strong><\/span>. It was set up without any funding to and it is just a repurposed old storage server, it can fail any time and it has to be replaced. We have created a replacement service act-rsync<\/span>.reading.ac.uk<\/strong> (at some stage we used the name poplar.act.reading.ac.uk). It is a VM (virtual machine) and it will be more reliable.\u00a0 Formally, the cost of the VM should be charged to the projects using this custom service, but IT will provide it for free, to allow for smooth decommissioning of old poplar. Typically, the accounts actively using poplar will be migrated to the new systems, and new user accounts will not be created. We can make some exceptions tor users migrating from oak, but new users should rather use MFT<\/a> or act-ssh<\/a> for data transfers.<\/p>\n

General user’s instructions and service overview<\/h3>\n

Act-rsync<\/strong> is a service that allows rsync connections from off-campus machines authenticated just with ssh keys.\u00a0 If you did not set a passphrase on your private ssh key you can transfer data with rsync from off-campus machines, without entering any password. Normally, any off-campus connection requires two factor authentication.\u00a0 Allowing pasword-less connection from off-campus, as it is done here,\u00a0 makes the system vulnerable. This is why only rsync connection is available, and any other options for of connecting to this system are disabled. Also, UoR data storage volumes are not accessed directly via act-rsync, data can be only deposited in the internal storage, and then moved to UoR storage in a second step.\u00a0 Act-rsync’s local storage is exported to some ACT systems and mounted at \/storage\/shared\/act-rsync<\/span>. The exported storage can be used to access your data form the trusted network side, and to manage your ssh keys. Authorized_keys should be placed in \/storage\/shared\/act-rsync\/<username>\/authorized_keys<\/span> . Typically, username will be the same as on the old poplar, but with UID of a proper UoR group account, such that you will be able to access the files form the UoR group account. The data copied via rsync can be found at \/storage\/shared\/act-rsync\/<username>\/data\/<\/span>\u00a0, and\u00a0 the paths you specify to rsync will be relative to \/storage\/shared\/act-rsync\/<username>\/data<\/span>\u00a0. E.g.<\/p>\n

remotePC>$ rsync -avr my_source_dir\u00a0 <username>@act-rsync.reading.ac.uk:\/<\/pre>\n
will place my_source_dir<\/span>\u00a0 in \/storage\/shared\/act-rsync\/<username>\/data\/<\/span>. If your rsync command is:<\/p>\n
remotePC>$ rsync -avr my_source_dir\u00a0 <username>@act-rsync.reading.ac.uk:\/home\/<username>\/<\/pre>\n
and if you do not want to change it, you will just need to create the directory<\/p>\n
\/storage\/shared\/act-rsync\/<username>\/data\/home\/<username>\/<\/pre>\n
and you will find your data in<\/p>\n
\/storage\/shared\/act-rsync\/<username>\/data\/home\/<username>\/my_source_dir<\/pre>\n
Migration from poplar.reading to act-rsync<\/h3>\n
The system is planned for the migration to be straightforward (assuming you are using only rsync). This should be just a matter of changing the host address from poplar.reading.ac.uk to act-rsync.reading.ac.uk, maybe creating some directories in \/storage\/shared\/act-rsync\/<username>\/data\/<\/span>\u00a0, and all should work as before.\u00a0 To give an example, if you push data with the command<\/p>\n
remotePC>$ rsync -avr my_source_dir\u00a0 <username>@poplar.reading.ac.uk:\/home\/<username>\/<\/pre>\n
and pull data with the command<\/p>\n
racc-cron>$ rsync -avr <username>@poplar.reading.ac.uk:\/home\/<username>\/ \/storage\/silver\/<project>\/<\/pre>\n
You will just need to create a directory<\/p>\n
racc-cron>$ mkdir -p\u00a0 \/storage\/shared\/act-rsync\/<username>\/data\/home\/<username><\/pre>\n
and change your commands to<\/p>\n
remotePC>$ rsync -avr my_source_dir\u00a0 <username>@act-rsync.reading.ac.uk:\/home\/<username>\/<\/pre>\n
and<\/p>\n
racc-cron>$ rsync -avr <username>@act-rsync.reading.ac.uk:\/home\/<username>\/ \/storage\/silver\/<project>\/<\/pre>\n
Note that using rsync from the trusted network side it is possible but it is not necessary, as the storage is mounted directly. The second command can as well be replaced with:<\/p>\n
racc-cron>$ cp -a \/storage\/shared\/act-rsync\/<username>\/data\/home\/<username>\/* \/storage\/silver\/<project>\/<\/pre>\n","protected":false},"excerpt":{"rendered":"
This article is addressed to the users already using poplar for automated data transfers. It might be also useful to some users who will need to find an alternative for their automated data transfers currently running on oak.    Background on poplar.reading  Poplar.reading.ac.uk is a server located outside the University trusted network.<\/p>\n","protected":false},"author":16,"featured_media":0,"template":"","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"__cvm_playback_settings":[],"__cvm_video_id":"","_links_to":"","_links_to_target":""},"kbe_taxonomy":[81],"kbe_tags":[],"class_list":["post-5123","kbe_knowledgebase","type-kbe_knowledgebase","status-publish","hentry","kbe_taxonomy-related-articles"],"acf":[],"yoast_head":"\nPoplar replacement - Academic Computing Team<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/poplar-replacement\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Poplar replacement - Academic Computing Team\" \/>\n<meta property=\"og:description\" content=\"This article is addressed to the users already using poplar for automated data transfers. It might be also useful to some users who will need to find an alternative for their automated data transfers currently running on oak.  Background on poplar.reading Poplar.reading.ac.uk is a server located outside the University trusted network.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/poplar-replacement\/\" \/>\n<meta property=\"og:site_name\" content=\"Academic Computing Team\" \/>\n<meta property=\"article:modified_time\" content=\"2020-04-23T12:53:03+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/poplar-replacement\/\",\"url\":\"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/poplar-replacement\/\",\"name\":\"Poplar replacement - Academic Computing Team\",\"isPartOf\":{\"@id\":\"https:\/\/research.reading.ac.uk\/act\/#website\"},\"datePublished\":\"2019-10-02T18:42:39+00:00\",\"dateModified\":\"2020-04-23T12:53:03+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/poplar-replacement\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/poplar-replacement\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/poplar-replacement\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/research.reading.ac.uk\/act\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Knowledgebase\",\"item\":\"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Poplar replacement\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/research.reading.ac.uk\/act\/#website\",\"url\":\"https:\/\/research.reading.ac.uk\/act\/\",\"name\":\"Academic Computing Team\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/research.reading.ac.uk\/act\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/research.reading.ac.uk\/act\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/research.reading.ac.uk\/act\/#organization\",\"name\":\"University of Reading\",\"url\":\"https:\/\/research.reading.ac.uk\/act\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/research.reading.ac.uk\/act\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/research.reading.ac.uk\/act\/wp-content\/uploads\/sites\/2\/2017\/08\/cropped-University_of_Reading_shield-1.png\",\"contentUrl\":\"https:\/\/research.reading.ac.uk\/act\/wp-content\/uploads\/sites\/2\/2017\/08\/cropped-University_of_Reading_shield-1.png\",\"width\":512,\"height\":512,\"caption\":\"University of Reading\"},\"image\":{\"@id\":\"https:\/\/research.reading.ac.uk\/act\/#\/schema\/logo\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Poplar replacement - Academic Computing Team","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/poplar-replacement\/","og_locale":"en_GB","og_type":"article","og_title":"Poplar replacement - Academic Computing Team","og_description":"This article is addressed to the users already using poplar for automated data transfers. It might be also useful to some users who will need to find an alternative for their automated data transfers currently running on oak.  Background on poplar.reading Poplar.reading.ac.uk is a server located outside the University trusted network.","og_url":"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/poplar-replacement\/","og_site_name":"Academic Computing Team","article_modified_time":"2020-04-23T12:53:03+00:00","twitter_card":"summary_large_image","twitter_misc":{"Estimated reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/poplar-replacement\/","url":"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/poplar-replacement\/","name":"Poplar replacement - Academic Computing Team","isPartOf":{"@id":"https:\/\/research.reading.ac.uk\/act\/#website"},"datePublished":"2019-10-02T18:42:39+00:00","dateModified":"2020-04-23T12:53:03+00:00","breadcrumb":{"@id":"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/poplar-replacement\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/research.reading.ac.uk\/act\/knowledgebase\/poplar-replacement\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/poplar-replacement\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/research.reading.ac.uk\/act\/"},{"@type":"ListItem","position":2,"name":"Knowledgebase","item":"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/"},{"@type":"ListItem","position":3,"name":"Poplar replacement"}]},{"@type":"WebSite","@id":"https:\/\/research.reading.ac.uk\/act\/#website","url":"https:\/\/research.reading.ac.uk\/act\/","name":"Academic Computing Team","description":"","publisher":{"@id":"https:\/\/research.reading.ac.uk\/act\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/research.reading.ac.uk\/act\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/research.reading.ac.uk\/act\/#organization","name":"University of Reading","url":"https:\/\/research.reading.ac.uk\/act\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/research.reading.ac.uk\/act\/#\/schema\/logo\/image\/","url":"https:\/\/research.reading.ac.uk\/act\/wp-content\/uploads\/sites\/2\/2017\/08\/cropped-University_of_Reading_shield-1.png","contentUrl":"https:\/\/research.reading.ac.uk\/act\/wp-content\/uploads\/sites\/2\/2017\/08\/cropped-University_of_Reading_shield-1.png","width":512,"height":512,"caption":"University of Reading"},"image":{"@id":"https:\/\/research.reading.ac.uk\/act\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/kbe_knowledgebase\/5123","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/kbe_knowledgebase"}],"about":[{"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/types\/kbe_knowledgebase"}],"author":[{"embeddable":true,"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/users\/16"}],"version-history":[{"count":5,"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/kbe_knowledgebase\/5123\/revisions"}],"predecessor-version":[{"id":5699,"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/kbe_knowledgebase\/5123\/revisions\/5699"}],"wp:attachment":[{"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/media?parent=5123"}],"wp:term":[{"taxonomy":"kbe_taxonomy","embeddable":true,"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/kbe_taxonomy?post=5123"},{"taxonomy":"kbe_tags","embeddable":true,"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/kbe_tags?post=5123"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}