Knowing a few ssh tricks will benefit any system administrator, network engineer or security professional.

First The Basics

Breaking down the SSH Command Line

The following ssh example command uses common parameters often seen when connecting to a remote SSH server.

localhost:~$ ssh -v -p 22 -C youruser@remoteserver

• -v : Print debug information, particularly helpful when debugging an authentication problem. Can be used multiple times to print additional information.
• -p 22 : Specify which port to connect to on the remote SSH server. 22 is not required as this is the default, but if any other port is listening connect to it using the -p parameter. The listening port is configured in the sshd_config file using the Port 2222 format.
• -C : Compression is enabled on the connection using this parameter. If you are using the terminal over a slow link or viewing lots of text this can speed up the connection as it will compress the data transferred on the fly.
• youruser@ : The string before the @ symbol denotes the username to authenticate with against the remote server. Leaving out the user@ will default to using the username of the account you are currently logged in to (~$ whoami). User can also be specified with the -l parameter.
• remoteserver : The hostname that the ssh is connecting to, this can be a fully qualified domain name, an IP address or any host in your local machines hosts file. To connect to a host that resolves to both IPv4 and IPv6 you can specify add a parameter -4 or -6 to the command line so it resolves correctly.

Each of the above a parameters are optional apart from the remoteserver.

Using a Configuration File

While many users are familiar with the sshd_config file, there is also a client configuration file for the ssh command. This defaults to ~/.ssh/config but can also be specified as a parameter with the -F option.

Host remoteserver
     HostName remoteserver.yourhost.io
     User youruser
     Port 2113
     IdentityFile /home/test/.ssh/remoteserver.private_key

Host *
     Port 2223

In the above example ssh configuration file you can see that there are two Host entries. The first is a specific host entry with Port 2112 configured, as well as a custom IdentifyFile and username.

The second is a wildcard value of * that will match all hosts. Note that the first configuration option found will be used, so the most specific should be at the top of the configuration.

More information is found in the man page (man ssh_config).

The configuration file can save a lot of typing by including advanced configuration shortcuts any time a connection is made to particular hosts.

Copy Files over SSH with SCP

The ssh client comes with two other very handy tools for moving files around over an encrypted ssh connection. The commands are scp and sftp, see the examples below for basic usage. Note that many parameters for the ssh can be applied to these commands also.

localhost:~$ scp mypic.png youruser@remoteserver:/media/data/mypic3.png

In this example the file mypic.png was copied to the remoteserver to file system location /media/data and was renamed to mypic3.png.

Don’t forget the difference in the port parameter. This is a gotcha that hits everyone using scp on the command line. The port parameter is -P not -p as it is in the ssh client.!. You will forget, but don’t worry everyone does.

For those familiar with command line ftp, many of the commands are similar when using sftp. You can push, put and ls to your hearts desire.

sftp youruser@remoteserver

Practical Examples

In many of these examples we could achieve the result using a number of methods. As in all our tutorials and example command sheets, the focus is practical examples that get the job done.

1. SSH Tunnel (port forward)

In its simplest form an SSH tunnel opens a port on your local system that connects through to another port at the other end of the tunnel.

localhost:~$ ssh  -L 9999:127.0.0.1:80 user@remoteserver

Lets break down the -L parameter. Think of -L as the Local listening side. So in our example above the port 9999 is listening on localhost and port forwards through to port 80 on remoteserver, note that the 127.0.0.1 refers to localhost on the remote server!

Lets take it up a notch. In this following example the port that is listening can be connected to from other hosts on the local network.

localhost:~$ ssh  -L 0.0.0.0:9999:127.0.0.1:80 user@remoteserver

In these examples the port we are connecting is a listening web server. It could also be a proxy server or any other TCP service.

2. SSH Tunnel Forward to Secondary Remote host

We can use the same options seen above to have the tunnel connect to another service running on a secondary system from the remote server.

localhost:~$ ssh  -L 0.0.0.0:9999:10.10.10.10:80 user@remoteserver

In this example we are forwarding the tunnel from remoteserver to the web server running on 10.10.10.10. The traffic from remoteserver -> 10.10.10.10 is no longer within the ssh tunnel. The web server on 10.10.10.10 will see remoteserver as the source of the web requests.

3. SSH Reverse Tunnel

In this scenario we want to setup a listening port on the remote server that will connect back to a local port on our localhost (or other system).

localhost:~$ ssh -v -R 0.0.0.0:1999:127.0.0.1:902 192.168.1.100 user@remoteserver

With this ssh session established a connection to the remoteserver port 1999 will be forwarded to port 902 on our local client.

4. SSH Reverse Proxy

In this case we are establishing a SOCKS proxy with our ssh connection, however the proxy is listening at the remote server end. With connections to that remote socks proxy now emerging from the tunnel as traffic originating from our localhost. Requires OpenSSH version 7.6+.

localhost:~$ ssh -v -R 0.0.0.0:1999 192.168.1.100 user@remoteserver

Troubleshooting Remote SSH Tunnels

If you are having trouble getting the remote SSH options to work, check with netstat which interface the listening port is attached too. Even though we have specified 0.0.0.0 in the above examples, if GatewayPorts is set to no in the sshd_config then the listener will only bind to localhost (127.0.0.1).

Security Warning

Note that when you are opening tunnels and socks proxies you may be exposing internal network resources to untrusted networks (like the Internet!). This can be a serious security risk so ensure you understand what is listening and what it has access too.

5. Establish a VPN over SSH

A common term amongst offensive security folks (pentesters / red teams / etc), is to pivot into a network. Once you have a connection established on one system that system becomes a gateway point for further access to the network. This is known as pivoting and enables lateral movement through the network.

We can use the SSH proxy for this and proxychains, however there are some limitations. For example we cannot use raw sockets, so Nmap SYN scans cannot be used to port scan the Internal network.

Using this more advanced VPN option we move the connectivity down to layer 3. We can then route traffic through the tunnel using standard network routing.

This technique uses ssh, iptables, tun interfaces and routing.

First we need these options set in the sshd_config. Since we are making interface changes on the remote system and the client system, we will need root privileges on both sides.

PermitRootLogin yes
PermitTunnel yes

Then we will establish our ssh connection using the parameter that requests tun devices be initialised.

localhost:~# ssh -v -w any root@remoteserver

Now you should have a tun device when you show interfaces (# ip a). Next step is to add IP addresses to the tunnel interfaces.

SSH Client Side:

localhost:~# ip addr add 10.10.10.2/32 peer 10.10.10.10 dev tun0
localhost:~# ip tun0 up

SSH Server Side:

remoteserver:~# ip addr add 10.10.10.10/32 peer 10.10.10.2 dev tun0
remoteserver:~# ip tun0 up

Now we should have a direct route to the other host (route -n and ping 10.10.10.10).

It is now possible to route any subnet through the other side host.

localhost:~# route add -net 10.10.10.0 netmask 255.255.255.0 dev tun0

On the remote side we need to enable ip_forward and iptables.

remoteserver:~# echo 1 > /proc/sys/net/ipv4/ip_forward
remoteserver:~# iptables -t nat -A POSTROUTING -s 10.10.10.2 -o enp7s0 -j MASQUERADE

Layer three VPN through an SSH tunnel working !!!

Any trouble, try tcpdump and ping to see where its broken. Since we are playing at layer 3 our icmp packets should be jumping through that tunnel.

6. Copy your SSH key (ssh-copy-id)

There are multiple ways to achieve this however this command is a shortcut that saves time. What does it actually do? This command replicates what you can also do manually. Copying the ~/.ssh/idrsa.pub (or the default) key from your system and adds it to an ~/.ssh/authorizedkeys file on the remote server.

localhost:~$ ssh-copy-id user@remoteserver

7. Run Command Remotely (non-interactive)

The ssh command can be chained to other commands for the usual piping fun. Add the command you want to run on the remote host as a final parameter in quotes.

localhost:~$ ssh remoteserver "cat /var/log/nginx/access.log" | grep badstuff.php

In this example the grep is being performed on the local system after the log file has been pushed across the ssh session. If the file is large it would be more efficient to run the grep on the remote side by enclosing the pipe and grep in the double quotes.

Another example performs the same function as the ssh-copy-id short cut in Tip 7.

localhost:~$ cat ~/.ssh/id_rsa.pub | ssh remoteserver 'cat >> .ssh/authorized_keys'

8. Remote Packet Capture & View in Wireshark

I grabbed this one from our tcpdump examples. Use it for a remote packet capture with the results feeding directly into your local Wireshark GUI.

:~$ ssh root@remoteserver 'tcpdump -c 1000 -nn -w - not port 22' | wireshark -k -i -

9. SSH Copy Folder from Local to Remote

A neat trick that compresses a folder using bzip2 (that’s the -j in the tar command), then extracts the bzip2 stream on the other side creating a duplicate of the folder on the remote server.

localhost:~$ tar -cvj /datafolder | ssh remoteserver "tar -xj -C /datafolder"

Copy remote folder to local tar archive

To go the other way, copying a remote folder to a local archive. Handy for quick backups of remote resources.

localhost:~$ ssh user@remoteserver "tar -jcf - /path/to/backup" > dir.tar.bz2

10. Copy files remotely with rsync and SSH

Using the rsync has many advantages over scp, if periodically need to backup a directory, large numbers of files or very large files it should be used. It has the ability to recover from failed transfers and only copy differences between two locations saving bandwidth and time.

The example here uses gzip compression (-z) and archive mode (-a) that includes recursive copy.

:~$ rsync -az /home/testuser/data remoteserver:backup/

11. SSH over Tor Network

The anonymised Tor Network can tunnel SSH traffic by using the torsocks command. The following command will proxy the ssh connection through the Tor network.

localhost:~$ torsocks ssh myuntracableuser@remoteserver

Torsocks will use the localhost port 9050 to proxy traffic. As always when using tor serious consideration must be taken to understand what traffic is being tunnelled and other operational security (opsec) concerns. Where are your DNS requests going?

Wrapping Up

These ssh examples, tips and commands are intended to give you a good starting point; additional detail on each of the commands and capabilities is available using the man pages (man ssh, man sshconfig, man sshdconfig).

Being able to reach out and run commands on systems anywhere in the world has always fascinated me. By developing your skills with tools such as ssh you will become more productive and effective at whatever game you play.

Thanks for reading and if you have any comments or suggestions please drop me a note using the contact form. Have fun!