Knowledge Base

ssh iconact-ssh – connection caching for automated data transfers from Unix

This article will be updated to match with other act-ssh articles. The information provided here is correct, but ssh multiplexing is now also explained in the article:  act-ssh for Linux and Mac users – step by step guide

Users relying on scripted data transfers via oak can replace using ssh-agent with equally convenient connection multiplexing, as described at the end of this article.  In rare cases where connection multiplexing cannot be used (in the context of migrating from oak), we will discuss individual solutions, using act-rsync, or Managed File Transfer service, or even individual authentication policy exceptions on act-ssh. If there is large demand for help with setting this up we will organize a tutorial session (for users already using ssh for data transfers).

This will be phased out, new users are not added to the group: Users connecting from UK academic institution with the domain name ending with ‘.ac.uk’ (other trusted institutions can be added as needed), and who rely on scripted data transfers that cannot be done with ssh connection caching, can apply to be added to the exception list (the security group act-ssh-academic-users), and use ssh key authentication without UoR password to connect from those institutions. By applying for the exception, those users commit to always use password protected private ssh keys (with ssh-agent, if needed).

Allowing automatic connections from your account on some off-campus machine is potentially putting your data at risk and compromising security of the whole campus network. Please use the following procedures responsibly, only on secure systems at trusted academic institutions, or on a personal machine which is used only by you. A separate data transfer options, MFT for sftp and scp and act-rsync for rsync,  are provided and suitable also in the case of less secure off-campus machines.

Background on the ssh-agent

When using ssh key authentication on oak users where asked to protect their private key with passphrase. Ssh keys that are not protected by a passphrase were only allowed to be used on legacy poplar, and now on act-rsync. Passphrase is independent from UoR password, it is just used to protect encrypted ssh key, such that in case the private ssh key is leaked out it cannot be used by a third party. It is the same logic as when a file is encrypted and protected with a password, e.g. you need to enter a password to open your pdf payslip. To avoid having to type the passphrase multiple times, ssh-agent requires you to enter the passphrase just once (it is needed to decrypt the key), and then runs in the background and keeps decrypted ssh key in memory, potentially for as long as the client machine is not restarted. Ssh-agent will not allow fully automated connections to act-ssh, because here, after the ssh key authentication, authentication with the UoR password is required.

Connection reusing (ssh multiplexing)

Openssh allows to create an authenticated ssh connection, put it in the background, and reuse it for further ssh sessions, without a need to authenticate again. The procedure will be similar to using ssh-agent. On a trusted off-campus system, you will need to set up the master connection and authenticate with one the two multi-factor authentication methods described above. Once this master connection is established, it stays in the background, even after you log out, and all your subsequent ssh connections from this client machine will not require further authentication. To set it up, you need to create a personal ssh client configuration file, the default location is  ~/.ssh/config, and put there the following lines

Host act-ssh
        HostName act-ssh.reading.ac.uk
        User <UoR login name>
        Port 2222
        ServerAliveInterval 30
        ServerAliveCountMax 3
        ControlPath ~/.ssh/controlmasters/%l_%r@%h:%p
        ControlMaster auto 
        ControlPersist yes

With this set up, and the directory ~/.ssh/controlmasters  created, with the right permissions (see below), once you open a connection to act-ssh for the first time and authenticate, a persistent background process and a control socket, at the location as specified in the configuration file, will be created automatically (with ‘ControlMaster auto’), and they will be reused for subsequent connections, without a need to authenticate again. The lines ‘ControlMaster auto’  and ‘ControlPersist yes’ are optional, and in some older versions of openssh client these options are not recognized. Without ‘ControlMaster auto’  the  background master connection is not created automatically, but it can be created manually. We recommend creating the master connection manually anyway, for better control whether it is created or not. We will show this procedure in the following example, on met-cluster (substitute mimas with your remote system, e.g. archer, jasmin, etc.). Met-cluster is intentionally not treated as a trusted host (requires two factor authentication) to allow testing. The option ‘User ‘ is handy if your user name on the off-campus client machine is different than your UoR user name. There is also a handy option ‘Port’ in case you want to use port 2222 and ssh key authentication. Adding the two ‘ServerAlive’ options might help with keeping the inactive master connection alive. On some systems you might need to use shorter ‘ControlPath ~/.ssh/controlmasters/%L_%r@%n:%p‘, instead of ‘ControlPath ~/.ssh/controlmasters/%l_%r@%h:%p’.

mimas$ mkdir .ssh/controlmasters
mimas$ chmod 700 .ssh/controlmasters
mimas$ cat .ssh/config
Host act-ssh
      HostName act-ssh.reading.ac.uk
      User <UoR login name> 
      Port 2222 
      ServerAliveInterval 30 
      ServerAliveCountMax 3
      ControlPath ~/.ssh/controlmasters/%l_%r@%h:%p   

‘ControlMaster auto’ and ‘ControlPersist’ are not available on met-cluster. We need to create the master connection by hand. The master connection will stay active after you log out. In this example we connect on default port 22, and we use Authenticator, not the ssh key, to make the two factor authentication step more obvious. Connection on port 2222 (port number can be specified in the configuration file) and uthentication with ssh key will work as well.

mimas$ ssh act-ssh -fNM
Verification code:
Password:
mimas$ ssh -O check act-ssh 
Master running (pid=1893406)
mimas$ exit
homePC$

We login to met-cluster again, confirm that the master connection is still active, and we connect to act-ssh without authentication.

homePC$ ssh met-cluster
mimas$ ssh -O check act-ssh
Master running (pid=1893406)
mimas$ ssh act-ssh
Last login: Sun Nov 3 15:17:51 2019 from mimas.nerc-essc.ac.uk
[qx901702@act-ssh ~]$

We can stop control master with the command ssh -O exit act-ssh 

mimas$ ssh -O check act-ssh
Master running (pid=1893406)
mimas$ ssh -O exit act-ssh
Exit request sent.
mimas$ ssh -O check act-ssh
Control socket connect(/home/users/qx901702/.ssh/controlmasters/mimas.nerc-essc.ac.uk:qx901702@act-ssh.reading.ac.uk:22): No such file or directory

Please remember to stop unused master connections.

Connection caching tips for Linux users working from home

This section is really intended for experienced Unix users, the procedures described here are not a service supported by the DTS. It is given as an informal advice and we are not able to provide individual support to users . The following example of the ~/.ssh/config file can be used as a source of inspiration by Linux or Mac users who can’t or don’t want to use VPN. In addition to connection caching such that you need to enter your password only once, when you log in to your home Linux machine, it sets up an local port forwarding ssh tunnel (here port 3389 for Windows Remote Desktop connection and port 4000 for nx.reading.ac.uk), and a SOCKS proxy tunnel which can be used with a Web browser, for connecting to nx.reading.ac.uk using the Enterprise NX Client for Linux and with other software which is able to connect via SOCKS proxy.

Host act-ssh
 HostName act-ssh.reading.ac.uk
 User qx901702
 Port 2222
 ServerAliveInterval 30
 ServerAliveCountMax 3
 ControlPath ~/.ssh/controlmasters/%l_%r@%h:%p
 ControlMaster auto 
 ControlPersist yes
 ForwardX11 yes
 DynamicForward localhost:4444
 Localforward localhost:33389 smpc115.rdg.ac.uk:3389
 Localforward localhost:4000 nx.reading.ac.uk:4000
Host cluster
 Hostname cluster.act.rdg.ac.uk
 ProxyCommand ssh -W %h:%p act-ssh -Y
 User qx901702
 ServerAliveInterval 30
 ServerAliveCountMax 3
 ControlPath ~/.ssh/controlmasters/%l_%r@%h:%p
 ControlMaster auto
 ControlPersist yes
 ForwardX11 yes

With this configuration in place one can use SOCKS v4 proxy on localhost:4444 for web browsing or for connection to NX. To connect to my Windows desktop I use the command:

pawel@homePC:~$ xfreerdp /u:qx901702 /v:localhost:33389 -f

The second part of the file sets up master connection to RACC using act-ssh as a proxy. With this setup we just type

pawel@homePC:~$ ssh cluster

and we are in. Another advantage is that reusing cached connection works really fast.