How To: Set up a reverse proxy

Setting up external access to servers inside your home network is possible by setting up reverse tunnels behind a reverse proxy.

You will need an external server that is publicly accessible by the Internet. This server can serve any number of private servers inside your network via a reverse proxy. I personally use a basic DigitalOcean droplet that costs $6 per month. You don’t need a lot of power for this server, but you want something that won’t max out your bandwidth or CPU limits, as that will introduce a lot of latency to your SSH tunnels and ultimately your applications.

You will need to configure two components to enable access to your home network. First, set up the reverse tunnel to tunnel traffic from the remote server to the local server. Then, configure the reverse proxy on the remote server to direct traffic to the reverse tunnel.

Definitions

External/Remote server: This is the server that is running outside of your network and is publicly accessible from the Internet.
Local/Private server: This is a server running inside of your home network. You can have multiple local servers, in which you would follow these instructions for each tunnel on each server.
Reverse tunnel: This is an SSH tunnel that is established from the server that the traffic is being forwarded to. This is the opposite of a traditional tunnel which is initiated from the server sending the traffic, hence the name reverse tunnel. Reverse tunnels are useful when the destination of the traffic is inaccessible from the source of the traffic, such as a home network that has inbound traffic blocked by an ISP.
Reverse proxy: This is a web server that listens for HTTP/s traffic and redirects (i.e. proxies) that traffic to a different endpoint. In this setup, the reverse proxy helps isolate your network and tunnel from potentially dangerous traffic on the Internet. A traditional proxy is a web server that processes outbound traffic, hence why one that processes inbound traffic is called a reverse proxy.

Reverse SSH Tunnel

The reverse tunnel creates an SSH session to a remote machine and tunnels traffic from the remote machine to the local machine. You will need one reverse tunnel for each port you want to forward traffic to. You can have as many tunnels as you need from the same machine.

Setting up the reverse tunnel

Use the following ssh command from the local server to create the reverse ssh tunnel:

/usr/bin/ssh -N -o ServerAliveInterval=60 -o ServerAliveCountMax=3 -o ExitOnForwardFailure=yes -R <remote-listening-port>:<local-forwarding-server>:<local-forwarding-port> -i <path-to-private-key> <remote-username>@<remote-server>

-N: This option tells SSH not to set up a full shell session for this connection. This will only establish the SSH connection for the tunnel.
-o ServerAliveInterval=60: This option sends a keep-alive signal through the tunnel every 60 seconds. You may need this option if the tunnel is closed due to inactivity.
-o ServerAliveCountMax=3: This option tells the connection to terminate when 3 keep-alive signals are not responded to. This will automatically close the tunnel if the remote machine has disconnected or crashed without closing the connection. Without this option, the tunnel may stay open but unconnected and will need to be manually restarted in order to reconnect to the remote server. In addition, when used in a systemd service (covered in a later section), this allows the tunnel to automatically restart itself.
-o ExitOnForwardFailure=yes: This option tells SSH to exit with a failure if it can’t establish the connection. This allows it to automatically retry when inside of a systemd service.
-R <remote-listening-port>:<local-forwarding-server>:<local-forwarding-port>: This is the option for establishing the reverse tunnel. The tunnel will be set up to listen on port <remote-listening-port> on the remote server, and all traffic that arrives at that port will be forwarded to <local-forwarding-server> on port <local-forwarding-port> in your home network.

Note that by default, the remote port is bound to localhost, so it is not accessible to the Internet. You can choose to bind it to a public address by specifying a remote server address to bind to (for example, -R 0.0.0.0:<remote-listening-port>:<local-forwarding-server>:<local-forwarding-port> would allow any traffic to access the tunnel) but this is generally highly discouraged because you’re opening a direct connection from the Internet into your home network. There are bots constantly scanning the Internet for open ports to exploit, and then once an application is compromised they will move laterally inside your home network to compromise the rest of your devices. It’s recommended to keep the tunnels bound to localhost and run them behind a reverse proxy, which will be covered in the next section.
-i <path-to-private-key>: This option specifies the path to the private key file to use to authenticate the connection. You can use a different form of authentication if you want, for example username and password, but it’s highly recommended not to have your username or password hard coded, and key pair authentication is easier to use for unattended services.
<remote-username>@<remote-server>: Finally, this is the username and server (either IP address or domain name) of the remote server. The user should be a special user used just for your SSH tunnels and have all other permissions and logon rights disabled for security. Instruction on how to set this user up will be covered in a later section. However, you can also use just a regular logon user for this, it’s just less secure.

This is an example that I am currently using to create a tunnel from port 3001 on a public DigitalOcean droplet to an instance of Jellyfin running on the same machine on port 8096:

/usr/bin/ssh -N -o ServerAliveInterval=60 -o ServerAliveCountMax=3 -o ExitOnForwardFailure=yes -R 3001:localhost:8096 -i /home/sshtunnel/.ssh/tunnel_rsa sshtunnel@cloud.michaelhumphrey.dev

Creating a service for the tunnel

If you run the reverse tunnel from the command line, it will only stay running as long as the command is running. If you close the terminal it’s running in, the reverse tunnel will terminate. You can always start the command and run it in the background, but then you may not notice if the tunnel shuts down, and is difficult to restart. To fix these issues, you can configure a systemd service to manage the tunnel for you. Below is a template you can use to create the service:

[Unit]
Description=<service-name>
After=network-online.target

[Service]
User=<username>
ExecStart=/usr/bin/ssh -N -o ServerAliveInterval=60 -o ServerAliveCountMax=3 -o ExitOnForwardFailure=yes -R <remote-listening-port>:<local-forwarding-server>:<local-forwarding-port> -i <path-to-private-key> <remote-username>@<remote-server>

RestartSec=5
Restart=always

[Install]
WantedBy=multi-user.target

Description=<service-name>: A short description of this tunnel.
After=network-online.target: Wait to start this service until after the network is online.
User=<username>: The user this service should run under. Ideally this would be a user that is only used to run the SSH tunnels and have no other permissions and logon rights disabled, but it can also be a regular user. See the next section for instructions on how to set up a user for the tunnel.
ExecStart=<ssh-command>: This is the command to be run when this service is started. See the previous section for instruction on how to set up the ssh command.
RestartSec=5: This option specifies that if this service stops for any reason, it should wait 5 seconds before restarting. You can set this to whatever you like, but it should be short enough to minimize downtime if the tunnel can be reestablished and long enough to let the network connection shutdown and cleanup its resources before trying again. For less critical application, you can set this upward of 30-60 seconds or more to avoid straining your server if the remote server becomes unavailable for a period of time. Don’t set this to anything less than two seconds, otherwise the service may be terminated for starting up too many times too quickly.
Restart=always: This option specifies that the service should always start up after it stops for any reason. You can also set this option to on-failure to only start back up if it crashed.

Creating a user for the tunnel

To minimize the attack vector of exposing a tunnel to your home network from the Internet, it is recommended to set up a dedicated user with minimal permissions to run the tunnel. This ensures that even if that account is compromised, attackers still don’t have access to any other systems. You only need one user for all of your tunnels to use. In these examples, I’m using sshtunnel as the username, but you can use any username you prefer.

On the remote server

First, create the user and required directories on the remote server:

sudo useradd -m -s /usr/sbin/nologin sshtunnel
sudo mkdir -p /home/sshtunnel/.ssh

useradd: This command creates the user that the tunnel will run under.
-m: This option creates a home directory for the new user. We will use this directory to store the scripts for running the tunnel and symlinking them to the appropriate system directories.
-s /usr/sbin/nologin: This option sets the user’s login shell to /usr/sbin/nologin. This is a special shell that disallows this user from logging into the system, but lets it run system services and create tunnels. This prevents this user from being commandeered by an attacker while still allowing us to authenticate with a minimum-permission account for setting up the tunnel.
mkdir This command create the directory where the user’s ssh credentials will live.
-p: This option creates all parent directories for this directory if they don’t already exist.

Next, ensure that the directories belong to the user we created, not the currently logged in user or root:

sudo chown -R sshtunnel:sshtunnel ~/sshtunnel/.ssh
sudo chmod 700 ~/sshtunnel/.ssh
sudo chmod 600 ~/sshtunnel/.ssh/authorized_keys

chown -R sshtunnel:sshtunnel ~/sshtunnel/.ssh: This command recursively sets all files and directories in ~/sshtunnel/.ssh to belong to the sshtunnel user and group.
chmod ...: These commands set the permissions of the directory to the minimum necessary. The ssh keys for login are stored in these directories, so it’s important to keep them secure.

On the local server

Next, set up the reverse tunnels on the local server to use the newly created user.

First, create ssh key pair for authentication:

sudo mkdir -p /etc/sshtunnel
sudo ssh-keygen -qN "" -f /etc/sshtunnel/id_rsa

mkdir -p /etc/sshtunnel: This command creates a new directory for the tunnel credentials to live.
ssh-keygen: This command creates a new key pair for authentication.
-q: This option silences the output of the ssh-keygen command. It is optional.
-N "": This option sets the passphrase of the key pair to an empty string, essentially disabling the password requirement to use it. This makes it easier to allow the automated services to use it without prompting for a password. However, it makes it all the more important to keep these keys private, as anyone can use them to gain access to your system.
-f /etc/sshtunnel/id_rsa: This option specifies the newly created directory to store the key pair in.

Next, modify the systemd service files for the tunnels to use the new user. Most importantly, that means setting these ssh options:

-i /etc/sshtunnel/ssh: This points ssh to the private key we just created for the sshtunnel user.
sshtunnel@<remote-server>: Set the username when connecting to the remote server to sshtunnel.

Finally, copy the public key /etc/sshtunnel/id_rsa.pub to the remote machine and add it to the authorized keys file in ~/sshtunnel/.ssh/authorized_keys. This will allow the remote server to recognize and authenticate the newly created user. Make sure you connect to the remote server with the new credentials manually from the command line so you can whitelist the remote server’s public key as well.

Reverse Proxy

The other component needed is a reverse proxy to direct traffic from the Internet into the private SSH reverse tunnel you set up. The reverse proxy can do a lot of useful things for you, including providing a HTTPS endpoint in front of your services, load balancing, authentication, etc.

Configuring the reverse proxy

I use NGINX for my reverse proxy, but you can really do this part with any web server. The configuration options will be different, but the theory is the same.

Here is how I have my reverse proxy set up to terminate HTTPS connections and forward through the reverse SSH tunnel:

server {
    server_name <external-domain-name>;
    client_max_body_size 500M;  # Only needed if you're expecting large payloads

    location / {
        proxy_pass http://127.0.0.1:<reverse-tunnel-remote-port>;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    listen 443 ssl; # managed by Certbot
    ssl_certificate <path-to-fullchain.pem>; # managed by Certbot
    ssl_certificate_key <path-to-privkey.pem>; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
    if ($host = <external-domain-name>) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    server_name <external-domain-name>;
    listen 80;
    return 404; # managed by Certbot
}

The second server block just redirects all HTTP requests for this domain to HTTPS. This makes it easier for any clients attempting to connect with HTTP first to be forced to use HTTPS. This block is automatically added when using the LetsEncrypt client.

The first server block handles all of the actual proxying.

server_name <external-domain-name>: This binds this server block to a particular domain. Different server blocks can be bound to different domains and ran simultaneously; NGINX will just check which domain a request is for and use the configs for the associated server block. If you don’t have this label, it will use it for all incoming requests, but then you would only be able to have one server block configuration for everything.
client_max_body_size 500M: By default NGINX limits incoming request payloads to 1 megabyte. This is fine for most applications, but if you need to set it to be larger (for example, Nextcloud or other backup solutions) you can use this option to set that limit higher. Note that sending excessively large payloads to overwhelm your server is a common denial-of-service attack and can result in exorbitant server expenses, so only use this option if you need it and set it to the lowest value that you actually expect to use.
location /: This block specifies the URL root. Setting the root to / will make this block handle all requests on the domain. You can specify a URL fragment if you want to run multiple services on the same domain. For example, you can have one service in a location /foo block and another service in a location /bar block, and then have different settings in each one. Then requests coming in at mydomain.com/foo will be sent to a different SSH tunnel than requests coming in to mydomain.com/bar.

While this configuration will allow NGINX to route subpath requests to different SSH tunnels, the applications running behind those tunnels will have to be configued to support such a configuration, as clients will still end the entire URL with the subpath in their HTTP requests. Not all applications support running on a subpath, so it’s up to you to decide which ones, if any, will run that way. I personally prefer to use subdomains to separate my services (e.g. foo.mydomain.com and bar.mydomain.com) as it’s easier and cleaner to support, but you do need full control over your domain to do so.
proxy_pass http://127.0.0.1:<reverse-tunnel-remote-port>: This option specifies where traffic in this block should be forwarded to. The 127.0.0.1 specifies it should be forwarded to the localhost, where the reverse tunnel is listening. <reverse-tunnel-remote-port> should be the same remote port you specified when setting up the reverse tunnel. We are connecting the reverse proxy to the tunnel here.
proxy_set_header ...: These configs are to properly pass request headers through the proxy to the application behind the proxy. Many services require these headers to be set properly, especially if you’re running security applications. Applications may not work, generate security errors, or report incorrect data if these are headers are not set correctly.
The remaining headers in the server block are to configure HTTPS. These should be automatically added when requesting a certificate through LetsEncrypt.

NGINX configuration files

The configuration for a reverse proxy should be saved with the filename that matches the domain it’s running on. For example, if your reverse proxy is running for the domain foobar.mydomain.com, then you would save the configuration in a file called foobar.mydomain.com. The directory for these configurations is by default /etc/nginx/sites-available. Then, when you’re finished editing the configuration file, create a symlink to the file in /etc/nginx/sites-enabled using the command :

ln -s <target-path> <link-name>

The target path is the full path to the file you want to symlink to (for example, /etc/nginx/sites-available/foobar.mydomain.com) and the link-name is the name you want the link to have (in most cases, it should be the same). When you are done, you can check the link by typing ls -l and you should see something like this:

lrwxrwxrwx 1 root root 45 Apr  6  2024 foobar.mydomain.com -> /etc/nginx/sites-available/foobar.mydomain.com

Once the symlink is created, your reverse proxy configuration is now ready to be used. First run the command nginx -t to test your configuration files. If there are any errors, fix them now. When the test returns successfully, you can reload the NGINX configs with the command systemctl reload nginx. Your reverse proxy is now live and directing HTTP traffic to your tunnel.

Alternatives to self-hosting

If you don’t have a publicly accessible server or don’t want to purchase a VPS (virtual public server) to host the external reverse proxy, there are other services that will do the equivalent of these steps for you. I have used Packet Riot in the past to forward traffic from the internet to my local network. Their free tier offers up to one tunnel, unlimited HTTP/s proxies for that tunnel, 1 TCP proxy for the tunnel, and up to 1 gigabyte of bandwidth per month. The next tier is only $5 / month and allows up to a terabyte of bandwidth. I like this service quite a bit, but ultimately had some intermittent issues that led me to pursue hosting my own reverse proxy since I already paid for a VPS.

Many self-hosters use ngrok as a reverse proxy for their services. It’s quite a bit more expensive but can handle almost everything for you. Tailwind is also a great option if you don’t need your services to be truly public, but want to access your services from your own devices from anywhere in the world. There’s countless other alternatives out there that will all do the same thing, but ultimately under the hood they function the same way as the tools described in this document.

Read more:

My first post