New: Academic Computing Drop-in Sessions
Academic computing drop-in sessions are now run by members of our team on Wednesday afternoons! Head to our drop-in info page for details & schedule....
Knowledge Base
- /
- /
- /
arc-ssh for Linux and Mac users – step by step guide
In this article we show how to set up a home Unix machine to connect to campus services via arc-ssh. Connection caching is discussed later in this article can be also useful in the case of scripted data transfer from off-campus systems. There is also the sibling article arc-ssh with MobaXterm – step by step guide. It might be useful to read the MobaXterm article for some extra background explanations. Here we discuss how to set up ssh key authentication, which is recommended for all users. We also mention ssh multiplexing and other tips and trick to automate everyday tasks, which is aimed at more experienced users.
The commands and the configuration options should be treated just as examples, they might need to be modified to work on your own machine. This document builds on what is already described in the general instructions for arc-ssh, but the goal is to present it as a consistent series of steps, suitable for this particular use case, e.g. we assume that you are working from home, you want to use ssh keys and your openssh version is fairly recent.
1. Setting up ssh keys – this replaces inconvenient one time codes and it is recommended for all arc-ssh users
First, we create an ssh key pair on the home machine. You can skip this step if you already have a public and private key pair in your ~/.ssh/ directory. The public key should be appended to the file ~/.ssh/authorized_keys in our UoR Unix home directory. Here, we will not do it by hand, but we will use the convenient ssh-copy-id command which will also automatically create missing files or directories and correct access permissions if necessary.
Let us create the ssh keys with the ‘ssh-keygen’ command as seen below. We will use the default file name id_rsa. In this case, we will also not protect this ssh key with a passphrase because here the ssh key is just one of the two authentication steps and we will need our UoR password anyway.
janedoe@homePC:~$ ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/home/janedoe/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/janedoe/.ssh/id_rsa Your public key has been saved in /home/janedoe/.ssh/id_rsa.pub The key fingerprint is: SHA256:Tc0MZiEuZS/9i2T6w/G3taoy5c+DeufkzHEEUm23C/0 janedoe@homePC The key's randomart image is: +---[RSA 3072]----+ | + =. .. | | + * =. o.| | . o +.+.o o| | . + ....o | | S + . ..o| | +.o ...E| | ..+ooo ..| | +o+Bo+..| | .*o=X+o | +----[SHA256]-----+ janedoe@homePC:~$
As we see in the output above, a pair of keys was created. The private key, id_rsa, should stay on your machine and it should be kept secret. The public key, id_rsa.pub, should be copied to the remote server. This is achieved with the command ‘ssh-copy-id’. As we need to login to arc-ssh in order to transfer the key, we need to use a one time code as the second authentication factor (in addition to the UoR password, which is always required). You can see the procedure in the screenshot below.
janedoe@homePC:~$ ssh-copy-id <UoR login>@arc-ssh.reading.ac.uk The authenticity of host 'arc-ssh.reading.ac.uk (134.225.114.116)' can't be established. ECDSA key fingerprint is SHA256:ZeDQ/ns8OcD4ZpWw0R+1NlZ2GEHup08+YdSG+TrOyRY. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys ****************************************************************************** * NOTICE TO USERS * * * * Use of this system is restricted to registered members of The University * * of Reading. Unauthorised access is a criminal offence under United * * Kingdom law. * ****************************************************************************** Password: The code received by email (timestamp 19:04:59): Number of key(s) added: 1 Now try logging into the machine, with: "ssh '<UoR login>@arc-ssh.reading.ac.uk'" and check to make sure that only the key(s) you wanted were added. janedoe@homePC:~$
Once the ssh keys are set up, we can connect to arc-ssh.reading.ac.uk and we will be authenticated using the ssh key and password, so that entering the one time code is not longer needed.
A successful login with password and ssh keys looks like this:
janedoe@homePC:~$ ssh <UoR login>@arc-ssh.reading.ac.uk -X ****************************************************************************** * NOTICE TO USERS * * * * Use of this system is restricted to registered members of The University * * of Reading. Unauthorised access is a criminal offence under United * * Kingdom law. * ****************************************************************************** Password: Authenticated with public key. Last login: Sun Nov 22 18:26:59 2020 from 185.141.206.219 [<UoR login>@arc-ssh ~]$
From arc-ssh we can connect to other servers, e.g. to the RACC:
[<UoR login>@arc-ssh ~]$ ssh cluster.act.rdg.ac.uk -X Last login: Sat Nov 21 20:30:43 2020 from arc-ssh.reading.ac.uk Welcome to the Reading Academic Computing Cluster (RACC) racc-login.act.rdg.ac.uk is opening a session on one of the login nodes The login nodes can be used for considerate interactive computing and batch job submissions. See https://research.reading.ac.uk/act/knowledgebase/academic-cluster-usage Warning: Permanently added '[racc-login-0-5.local]:36628' (ECDSA) to the list of known hosts. Last login: Sat Nov 21 20:30:44 2020 from racc-login.local Rocks 7.0 (Manzanita) Profile built 12:06 24-Sep-2019 Kickstarted 12:14 24-Sep-2019 [<UoR login>@racc-login-0-5 ~]$
The above can also be done in one line directly from your home machine, using the ssh jump host option ‘-J’:
janedoe@homePC:~$ ssh -J <UoR login>@arc-ssh.reading.ac.uk <UoR login>@cluster.act.rdg.ac.uk -X ****************************************************************************** * NOTICE TO USERS * * * * Use of this system is restricted to registered members of The University * * of Reading. Unauthorised access is a criminal offence under United * * Kingdom law. * ****************************************************************************** Password: Last login: Sun Nov 22 19:23:56 2020 from arc-ssh.reading.ac.uk Welcome to the Reading Academic Computing Cluster (RACC) racc-login.act.rdg.ac.uk is opening a session on one of the login nodes The login nodes can be used for considerate interactive computing and batch job submissions. See https://research.reading.ac.uk/act/knowledgebase/academic-cluster-usage Warning: Permanently added '[racc-login-0-5.local]:42347' (ECDSA) to the list of known hosts. Last login: Sun Nov 22 19:23:57 2020 from racc-login.local Rocks 7.0 (Manzanita) Profile built 12:06 24-Sep-2019 Kickstarted 12:14 24-Sep-2019 [<UoR login>@racc-login-0-5 ~]$
2. Personal ssh configuration file – for more experienced users
Users can create their personal ssh client configuration file in ~/.ssh/config. We will use this file to automate many aspects of ssh connections via arc-ssh to achieve similar effects to those the Windows users enjoy if they follow the MobaXterm instructions. We can select different configuration options for specific hosts or groups of hosts. The following example contains a configuration that might be useful for users connecting from home and working on UoR Linux systems.
Host *
ServerAliveInterval 30
ServerAliveCountMax 3
# you might need to create the directory ~/.ssh/controlmasters/
ControlPath ~/.ssh/controlmasters/%l_%r@%h:%p
ControlMaster auto
ControlPersist yes
ForwardX11 yes
ForwardX11Trusted no
ForwardX11Timeout 500h
ForwardAgent yes
Host arc-ssh
HostName arc-ssh.reading.ac.uk
User <UoR_username>
# SOCKS proxy tunnel for NX and web browsing
DynamicForward 4444
# example: remote desktop to Windows machine on campus (commented out)
# Localforward 33389 <uor-campus-pc.rdg.ac.uk>:3389
# example: tunnel for NX (redundant example, we use SOCKS instead, commented out
# Localforward 4040 nx.reading.ac.uk:4000
Host !arc-ssh.reading.ac.uk *.rdg.ac.uk *.reading.ac.uk
User <UoR username>
ProxyJump arc-ssh
#older systems might not have ProxyJump, then use:
#ProxyCommand ssh arc-ssh -W %h:%p
The first group ‘*’ contains configuration options that will be applied to all hosts. The options ‘ServerAliveInterval’ and ‘ServerAliveCountMax’ will prevent closing temporarily inactive connections, e.g. when your terminal window is in the background while you have another active window open or if you are running a command that takes a long time to complete. The three options ‘ControlPath’, ‘ControlMaster’, and ‘ControlPersist’ are there to create and maintain a master connection (ssh multiplexing). This allows us to speed up subsequent connections to the same host. Because all subsequent connections will share the same master connection, there will be no need to enter the password again and starting a connection will be very fast, as no connection initialization and authentication will be needed. For these options to work you might need to create the directory ~/.ssh/controlmasters/.
janedoe@homePC:~$ mkdir /home/janedoe/.ssh/controlmasters
Finally, we typically will want to enable X11 forwarding, to be able to run GUI applications on the server and display the application window on the local machine, so we globally set ‘ForwardX11’ to ‘yes’.
The second part of the file contains options for the connection to arc-ssh. The first line ‘Host arc-ssh’ sets an alias, the name used here could be anything, but using the proper hostname makes sense. The full host name with domain component is defined in the next line. From now on we can use the short name arc-ssh and ssh will know that we want to connect to arc-ssh.reading.ac.uk using the configuration options set up in this section of the file. If you use a slightly different name that refers to the same host e.g. the full DNS name, the master connection will not be used. If you have a different user name on your home machine, it is convenient to specify the UoR user name here after ‘User’. If we just stop right here, we already have a very convenient set up. From now on, we can connect to arc-ssh just by typing ‘ssh arc-ssh’, not ‘ssh -X -o ServerAliveInterval=30 -o ServerAliveCountMax=3 <UoR login>@arc-ssh.reading.ac.uk’, and we will need to authenticate to arc-ssh only when making the connection for the first time. Here, we connect for the first time. The master connection doesn’t yet exist, so we authenticate with with UoR password and ssh key:
janedoe@homePC:~$ ssh arc-ssh ****************************************************************************** * NOTICE TO USERS * * * * Use of this system is restricted to registered members of The University * * of Reading. Unauthorised access is a criminal offence under United * * Kingdom law. * ****************************************************************************** Password: Authenticated with public key. Last login: Sun Nov 22 19:02:54 2020 from 185.141.206.219 [qx901702@arc-ssh ~]$ exit logout Shared connection to arc-ssh.reading.ac.uk closed. janedoe@homePC:~$
After the first login, the persistent master connection will run in the background, we can check this with the command ‘ssh -O check arc-ssh’. We can also see that there is a connection socket in our .ssh/controlmasters/ directory
janedoe@homePC:~$ ssh -O check arc-ssh Master running (pid=2955) janedoe@homePC:~$ ls -l .ssh/controlmasters/ total 0 srw------- 1 janedoe janedoe 0 Nov 22 19:57 homePC_<UoR login>@arc-ssh.reading.ac.uk:22
Subsequent connections will reuse the master connection and will not require a login:
janedoe@homePC:~$ ssh arc-ssh Last login: Sun Nov 22 19:57:53 2020 from 82.1.205.94 [<UoR login>@arc-ssh ~]$
If something doesn’t work, or when we are just curious what is happening behind the scenes, we can add the option ‘-vvv’ for increased verbosity to the ssh command.
Let’s skip the tunneling options in ~/.ssh/config for now, and move on to ‘Host *.rdg.ac.uk *.reading.ac.uk’. Here, we define the connection to all hosts in the UoR domain (excluding arc-ssh.reading.ac.uk which is listed above). The line ‘ProxyJump arc-ssh’ will automate connecting via arc-ssh as a jump host, so adding -J explicitly will not be needed.
janedoe@homePC:~$ ssh cluster.act.rdg.ac.uk Last login: Sun Nov 22 19:25:44 2020 from arc-ssh.reading.ac.uk Welcome to the Reading Academic Computing Cluster (RACC) racc-login.act.rdg.ac.uk is opening a session on one of the login nodes The login nodes can be used for considerate interactive computing and batch job submissions. See https://research.reading.ac.uk/act/knowledgebase/academic-cluster-usage Warning: Permanently added '[racc-login-0-5.local]:44842' (ECDSA) to the list of known hosts. Last login: Sun Nov 22 19:25:45 2020 from racc-login.local Rocks 7.0 (Manzanita) Profile built 12:06 24-Sep-2019 Kickstarted 12:14 24-Sep-2019 [<UoR login>@racc-login-0-5 ~]$
Note that the X11 forwarding option is switched on and that the correct UoR login name is used because they are defined in the configuration file.
In a similar fashion you can add entries for other hosts, also for external sites like JASMIN.
Now let’s discuss the ssh tunnels created in ~/.ssh/config file, in the arc-ssh section. The line ”DynamicForward localhost:4444″ is quite simple and quite powerful. It sets up a SOCKS proxy on arc-ssh and tunnels it to port 4444 on localhost. Now we can point any SOCKS aware application to localhost port 4444, and connect to campus services via arc-ssh. Two examples are the NoMachine Enterprise Client for Linux and Firefox.
In the case of NoMachine, you just point it to nx.reading.ac.uk as usual, then click ‘Advanced’, tick ‘Use proxy connection’ and click the corresponding ‘Settin’g button. There you just tick ‘Manual proxy configuration’, select SOCKS proxy and point it to localhost port 4444.
In Firefox, we go to Preferences->Network Settings and similar to NX, we set it to ‘Manual proxy configuration’, SOCKS v5, and localhost port 4444. Here it is important to tick ‘Proxy DNS when using SOCKS v5’, so we can use UoR DNS and resolve internal UoR IP addresses (many ARC IP addresses are like that). We might also be interested to use ‘No Proxy for’, and e.g. add *.com (or better not this one as this will include Microsoft sites, e.g. web Outlook, and they will require MFA if connecting from off-campus) and *.co.uk, such that connections to sites like youtube.com, www.netflix.com, or bbc.co.uk will not be unnecessarily routed via the proxy, i.e. via the UoR network. In general, one needs to be careful with this to not route their web browsing traffic via arc-ssh when not needed.
Another option with Firefox is the FoxyProxy Firefox extension, which allows to toggle proxy on and off easily. You can also select for which sites the proxy should be used, e.g you can set it up so that the proxy is only used when you connect to *.rdg.ac.uk, i.e. to the UoR sites accessible only from the campus network.
The following lines define local port forwarding tunnels, for RDP connections to a Windows desktop and for NX (see Connecting to NX from Outside the University Campus). Most likely we will not need them, as connecting to Windows desktop is not the recommended way of remote working and NX is already configured using SOCKS proxy, but we include those as examples how you can create automatic port forwarding tunnels in general.
3. Storage access
To access the unix storage, we can use arc-ssh or (without even setting up two factor authentication for arc-ssh) mft.act.reading.ac.uk with any GUI or command line sftp client software. If we want to make storage access more persistent and transparent, a good option is mounting your storage volumes with sshfs. Here we are on our personal machine, so we will set up the paths in the same locations as on the UoR systems (but mounting it that way, i.e. for one user, in a shared location on a shared machine would not be elegant).
The commands below show how to mount UoR storage volumes with sshfs:
janedoe@homePC:~$ sudo mkdir /storage/ janedoe@homePC:~$ sudo mkdir /dfs/ janedoe@homePC:~$ sudo mkdir -p /home/users/<UoR login> janedoe@homePC:~$ sshfs arc-ssh:/storage /storage janedoe@homePC:~$ sshfs arc-ssh:/dfs/<UoR login> /dfs janedoe@homePC:~$ sshfs arc-ssh:/home/users/<UoR login> /home/users/<UoR login>
And these are the effects:
janedoe@homePC:~$ ls /storage/ basic gold shared silver janedoe@homePC:~$ ls /dfs collabs ndrive research xdrive janedoe@homePC:~$ ls /home/users/<UoR login>/ <Jane Doe's UoR Unix home directory content>
Alternatively, the directories can be created in user’s home directory and sudo access is not needed.
NX authorization code