From a7b8edcf866269a8bc389abd411e6f7e0cfe0252 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 21:37:50 -0500 Subject: Document jail networking on a single public IP The --ip same-subnet rule from earlier is necessary but not sufficient once a host only has one public address: jails shouldn't get public IPs at all (only nginx needs one), but a private range aliased onto the same interface needs its own outbound NAT rule for jail builds to reach the internet - nothing upstream translates it the way a NAT'd virtual network (like the dev VM) does for free. Skipping it fails the same silent way as the out-of-subnet case: packets leave the jail, nothing answers. Co-Authored-By: Claude Sonnet 5 --- README.md | 35 +++++++++++++++++++++++++++++++++++ USER_GUIDE.md | 42 +++++++++++++++++++++++++++++++++++++++--- 2 files changed, 74 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index bff2a78..f3bf2b7 100644 --- a/README.md +++ b/README.md @@ -124,6 +124,41 @@ Without this, `skiff new-app` and manual `service jail start ` still work fine - both read `jail.conf.d/.conf` directly - so the gap stays hidden until the host reboots and no app jail comes back up. +### Jail networking on a single public IP + +Classic (non-VNET) jails share the host's network stack - `--ip` just adds +an address to the host's existing interface (see [First +deploy](#first-deploy)) - so on a host with only one public address, jails +shouldn't get public IPs at all. Only nginx needs one, listening on +80/443; every app's vhost `proxy_pass`es to its jail's own private +address. + +Alias a private range onto the same interface as the public IP, once: + +``` +# ifconfig vtnet0 inet 10.0.0.1/24 alias +``` + +and hand out jail `--ip`s from within it (`10.0.0.2`, `10.0.0.3`, ...). +Inbound just works from there - nginx reaching a jail is on-link delivery +on the same NIC, no gateway involved, the same same-subnet rule as always. + +Outbound doesn't, without one more step. A jail's build step needs real +internet access (module/package downloads - see +[Requirements](#requirements)), and a private range has nothing upstream +translating it to the public internet the way, say, a NAT'd virtual +network would - the host itself has to do that translation for its own +jails. Add a NAT rule to `/etc/pf.conf`: + +``` +nat on vtnet0 from 10.0.0.0/24 to any -> (vtnet0) +``` + +Skip this and jail builds fail exactly like the out-of-subnet case in +Requirements - packets leave the jail fine, `tcpdump` shows them, nothing +ever answers - just for a different reason (no return NAT, instead of no +route at all). + ### Nginx Add once to `/usr/local/etc/nginx/nginx.conf`, inside the `http` block: diff --git a/USER_GUIDE.md b/USER_GUIDE.md index 587a625..8cc62c3 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -368,7 +368,43 @@ bypassing `/etc/jail.conf` entirely. Only the boot-time, no-name form of stays invisible until the host reboots and no app jail comes back up on its own - see the [Troubleshooting](#troubleshooting) entry below. -#### 4. Nginx +#### 4. Jail networking on a single public IP + +Classic (non-VNET) jails share the host's network stack - `--ip` just +adds an address to the host's existing interface (see [Provisioning a new +app](#provisioning-a-new-app)) - so on a host with only one public +address, jails shouldn't get public IPs at all. Only nginx needs one, +listening on 80/443; every app's vhost `proxy_pass`es to its jail's own +private address. + +Alias a private range onto the same interface as the public IP, once: + +``` +# ifconfig vtnet0 inet 10.0.0.1/24 alias +``` + +and hand out jail `--ip`s from within it (`10.0.0.2`, `10.0.0.3`, ...). +Inbound just works from there - nginx reaching a jail is on-link delivery +on the same NIC, no gateway involved, the same same-subnet rule covered +above. + +Outbound doesn't, without one more step. A jail's build step needs real +internet access (module/package downloads - see +[Requirements](#requirements)), and a private range has nothing upstream +translating it to the public internet the way, say, a NAT'd virtual +network would - the host itself has to do that translation for its own +jails. Add a NAT rule to `/etc/pf.conf`: + +``` +nat on vtnet0 from 10.0.0.0/24 to any -> (vtnet0) +``` + +Skip this and jail builds fail exactly like the out-of-subnet case above +- packets leave the jail fine, `tcpdump` shows them, nothing ever answers +- just for a different reason (no return NAT, instead of no route at +all). + +#### 5. Nginx Add once to `/usr/local/etc/nginx/nginx.conf`, inside the `http` block: @@ -385,7 +421,7 @@ include /usr/local/etc/nginx/skiff.d/*.conf; Skiff only ever writes files under `skiff.d/` and reloads - it never touches `nginx.conf` itself. -#### 5. TLS (Let's Encrypt) +#### 6. TLS (Let's Encrypt) Skiff issues certificates itself via `certbot certonly` (webroot validation, never certbot's own `--nginx` plugin - Skiff owns @@ -416,7 +452,7 @@ every script under this directory after any successful renewal: # chmod +x /usr/local/etc/letsencrypt/renewal-hooks/deploy/reload-nginx ``` -#### 6. Install skiff +#### 7. Install skiff Cross-compile from any machine with Go, then copy the binary over: -- cgit v1.3