From 48d4c516c81dc3c1394ebea5ecd5aa588bc6960a Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 21:45:25 -0500 Subject: Flesh out the single-public-IP pf setup with what actually worked Replaces the sketch (bare ifconfig alias, isolated nat line) with what we verified end-to-end on a real VM: the alias persisted via rc.conf, a full pf.conf modeled on a plain host's firewall, and the pfctl/kldload steps needed to load and check it safely. Also documents a real bug hit along the way: `nat ... -> ($ext_if)` is ambiguous once the interface carries more than one address (the public IP and the private jail range both live on it) - pf can silently NAT through the wrong one. The public IP needs to be written explicitly. Co-Authored-By: Claude Sonnet 5 --- USER_GUIDE.md | 47 ++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 40 insertions(+), 7 deletions(-) (limited to 'USER_GUIDE.md') diff --git a/USER_GUIDE.md b/USER_GUIDE.md index 8cc62c3..f5b446c 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -377,9 +377,11 @@ address, jails shouldn't get public IPs at all. Only nginx needs one, listening on 80/443; every app's vhost `proxy_pass`es to its jail's own private address. -Alias a private range onto the same interface as the public IP, once: +Alias a private range onto the same interface as the public IP, persisted +so it survives a reboot: ``` +# sysrc ifconfig_vtnet0_alias0="inet 10.0.0.1/24" # ifconfig vtnet0 inet 10.0.0.1/24 alias ``` @@ -393,16 +395,47 @@ internet access (module/package downloads - see [Requirements](#requirements)), and a private range has nothing upstream translating it to the public internet the way, say, a NAT'd virtual network would - the host itself has to do that translation for its own -jails. Add a NAT rule to `/etc/pf.conf`: +jails. A full `/etc/pf.conf` for a skiff host looks like this - modeled on +a plain single-app host's firewall (default-deny inbound, explicit allowed +services), with one addition for the jail subnet: ``` -nat on vtnet0 from 10.0.0.0/24 to any -> (vtnet0) +ext_if = "vtnet0" +jail_net = "10.0.0.0/24" +tcp_services = "{ ssh, http, https }" + +nat on $ext_if from $jail_net to any -> 203.0.113.5 + +block in all +set skip on lo + +pass inet proto icmp all +pass inet6 proto icmp6 all +pass in on $ext_if proto tcp to port $tcp_services +pass out on $ext_if from any to any keep state +``` + +Write the host's actual public IP into the `nat` line (`203.0.113.5` +above is a placeholder) - not `-> ($ext_if)`. Once the interface carries +more than one address (the public one and the private jail range both +live on `$ext_if`), pf's dynamic-interface syntax can silently pick the +wrong one and NAT jail traffic to the *private* address instead, which is +just as unreachable from the internet as no NAT at all. + +Check syntax before loading - `pfctl -nf` needs the kernel module loaded +even for a dry run - then enable it the normal way: + +``` +# kldload pf +# pfctl -nf /etc/pf.conf +# sysrc pf_enable="YES" +# service pf start ``` -Skip this and jail builds fail exactly like the out-of-subnet case above -- packets leave the jail fine, `tcpdump` shows them, nothing ever answers -- just for a different reason (no return NAT, instead of no route at -all). +Skip the NAT step and jail builds fail exactly like the out-of-subnet case +above - packets leave the jail fine, `tcpdump` shows them, nothing ever +answers - just for a different reason (no return NAT, instead of no route +at all). #### 5. Nginx -- cgit v1.3