From 6040eff2d2b4618f19d4615d246ea6b0704acdf8 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 21:00:08 -0500 Subject: Add domain and TLS management via skiff domains add/remove/list Replaces the single server_name string in meta.json with a domain list (one primary, others redirect to it) and TLSDomains tracking what the app's certificate actually covers. Certs are issued/expanded via certbot certonly (webroot, never the --nginx plugin, since Skiff owns and regenerates skiff.d/*.conf itself) as one SAN cert per app, matching how a real multi-domain vhost is normally issued. Domain/TLS changes never touch the jail - no restart, no health check - since they only affect nginx and the certificate. `add` is safe to retry if DNS isn't ready yet; `remove` never calls certbot at all, so it can't fail on an ACME/network problem. Also updates README/USER_GUIDE for the new command surface, one-time TLS host setup (certbot register, renewal cron, reload hook), the --ip same-subnet requirement for jails to reach the network at build time, and the actual base-image provisioning sequence (pkg needs an explicit bootstrap after bsdinstall; versioned port names like go125 need a default symlink) - all found and fixed live while getting this working end-to-end on a real FreeBSD VM. Co-Authored-By: Claude Sonnet 5 --- USER_GUIDE.md | 149 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 145 insertions(+), 4 deletions(-) (limited to 'USER_GUIDE.md') diff --git a/USER_GUIDE.md b/USER_GUIDE.md index fe78bd0..63d6517 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -224,12 +224,23 @@ this in a hurry - it's more destructive than "undo." ``` # pkg install nginx git-lite ``` +- `certbot` - only needed if you'll route real domains via `skiff domains`: + ``` + # pkg install py312-certbot + ``` + (the package name tracks the default Python version, so this may drift) - Whatever your apps' Makefiles actually invoke to build - `go`, `python3`, `rustc`, `node`, etc. - baked into the base jail image (see [ZFS layout](#1-zfs-layout) below); builds run *inside* the jail via `jexec`, not on the host. Skiff itself doesn't care which; `make` is already in FreeBSD base (bmake), so it's never something you need - to install separately. + to install separately. Watch for versioned port names - FreeBSD's + `go125`, `python312`, etc. install as that literal binary, not a bare + `go`/`python3`; symlink a default into the base image's + `/usr/local/bin` if a Makefile invokes the unversioned name. The build + step also needs real outbound network access from inside the jail + (module/package downloads) - see the `--ip` note under [Provisioning a + new app](#provisioning-a-new-app). - `jail_enable="YES"` in `/etc/rc.conf`: ``` # sysrc jail_enable=YES @@ -246,6 +257,42 @@ this in a hurry - it's more destructive than "undo." # zfs create zroot/skiff # zfs create -o mountpoint=/skiff/base zroot/skiff/base # bsdinstall jail /skiff/base +``` + +`bsdinstall jail` only lays down the bare base system - it ships `pkg(8)`'s +bootstrap stub but never actually runs it, so a jail cloned from this alone +has no working `pkg` of its own (files can still be `pkg -c`-installed into +it from the host, but nothing *inside* the jail can manage them, and +`jexec pkg ...` fails with "the package management tool is not yet +installed"). Bootstrap pkg into the image itself, before snapshotting: + +``` +# mount -t devfs devfs /skiff/base/dev +# env ASSUME_ALWAYS_YES=yes pkg -c /skiff/base bootstrap -f +``` + +(`-f` because the stub already counts as "pkg is present" otherwise; devfs +is needed for pkg's own network/TLS calls inside a bare chroot - `fetch(1)` +wants `/dev/null` and `/dev/random`.) Now install whatever your apps' +Makefiles actually need to build: + +``` +# pkg -c /skiff/base install -y go125 +``` + +Watch for versioned port names - FreeBSD installs `go125` as literally +`go125`, not `go`, so a Makefile invoking the bare command needs a default +symlinked in: + +``` +# ln -sf go125 /skiff/base/usr/local/bin/go +# ln -sf gofmt125 /skiff/base/usr/local/bin/gofmt +``` + +Unmount devfs and snapshot once the image can actually build and run apps: + +``` +# umount /skiff/base/dev # zfs snapshot zroot/skiff/base@ready # zfs create -o mountpoint=none zroot/skiff/appbase # zfs create -o mountpoint=none zroot/skiff/appdata @@ -337,7 +384,38 @@ include /usr/local/etc/nginx/skiff.d/*.conf; Skiff only ever writes files under `skiff.d/` and reloads - it never touches `nginx.conf` itself. -#### 5. Install skiff +#### 5. TLS (Let's Encrypt) + +Skiff issues certificates itself via `certbot certonly` (webroot +validation, never certbot's own `--nginx` plugin - Skiff owns +`skiff.d/*.conf` and regenerates it on every domain/deploy change, and +would fight any other writer of that file). Register the certbot account +once: + +``` +# certbot register --agree-tos -m you@example.com +``` + +Renewal rides on FreeBSD's own `periodic(8)`, the same as any other +certbot install - Skiff runs no daemon or scheduler of its own for this, +and `certbot renew` is host-wide by design, so one cron job covers every +app's certificates regardless of when each was issued: + +``` +# sysrc -f /etc/periodic.conf weekly_certbot_enable="YES" +``` + +`certonly` mode doesn't reload nginx after a renewal the way certbot's +installer plugins do, so add that as a deploy hook once - certbot runs +every script under this directory after any successful renewal: + +``` +# mkdir -p /usr/local/etc/letsencrypt/renewal-hooks/deploy +# printf '#!/bin/sh\nnginx -s reload\n' > /usr/local/etc/letsencrypt/renewal-hooks/deploy/reload-nginx +# chmod +x /usr/local/etc/letsencrypt/renewal-hooks/deploy/reload-nginx +``` + +#### 6. Install skiff Cross-compile from any machine with Go, then copy the binary over: @@ -350,7 +428,7 @@ $ scp skiff root@host:/tmp/skiff ### Provisioning a new app ``` -# skiff new-app --name myapp --ip 10.0.0.5 --port 8080 --server-name myapp.example.com +# skiff new-app --name myapp --ip 192.168.1.50 --port 8080 ``` This clones the base dataset, writes `/etc/jail.conf.d/myapp.conf` and @@ -374,6 +452,58 @@ from the base image, so mail sent from inside the jail (e.g. by cron or the app itself) is attributed to `myapp.skiff.local` instead of the base image's own hostname. +**`--ip` must actually route on the host's `--iface` interface** - in +practice, an address in the same subnet as the host's own primary +address on that interface, not an arbitrary block like `10.0.0.0/8`. +Classic (non-VNET) jails share the host's routing table, so an +out-of-subnet address can still send packets - they just may never get a +response, silently, if whatever sits upstream (a NAT gateway, a +virtualized bridge) only answers its own subnet. This didn't matter +before builds moved inside the jail (see [Requirements](#requirements)); +now every deploy's build step needs real outbound network access from +that exact address (module downloads, `pkg install`, etc), not just +inbound reachability for the app itself. + +### Domains and TLS + +`new-app` only provisions the jail - nothing is routed until you add a +domain: + +``` +# skiff domains add --app myapp --primary myapp.example.com +# skiff domains add --app myapp www.myapp.example.com +# skiff domains list --app myapp +# skiff domains remove --app myapp www.myapp.example.com +``` + +`add` writes the hostname to the app's `meta.json`, routes it over plain +HTTP immediately (so the ACME HTTP-01 challenge below has somewhere to +land), then requests or expands a single certificate covering every +domain currently registered for the app, named after whichever is +primary - matching how a real multi-domain vhost is normally issued (one +SAN cert), not a cert per hostname. + +If DNS isn't pointed at the app's `--ip` yet, certificate issuance fails +and the command exits non-zero - the app stays reachable over HTTP, and +`add` is safe to re-run once DNS is fixed, exactly like retrying a failed +deploy (see [If the health check +fails](#if-the-health-check-fails)). Nothing about a failed `add` is +destructive. + +The first domain added for an app becomes primary automatically; +`--primary` reassigns it explicitly. Every non-primary domain +301-redirects to the primary, over HTTP and HTTPS alike, once its own +hostname is covered by the certificate. + +`remove` only updates routing - unlike `add`, it never calls out to +certbot, so a DNS or network problem can't make it fail. It's refused if +you try to remove the current primary while other domains remain - +reassign primary first (`skiff domains add --primary `). + +Domain/TLS changes never restart the jail and never run a health check - +unlike a code deploy or `skiff config`, they don't touch the running app +at all, only nginx and the certificate. + ### Rollback ``` @@ -452,6 +582,10 @@ Carried over from the project's own README, plus what testing surfaced: steps - not automated. - **`skiff rollback -r` is destructive** (see above) - there is currently no confirmation prompt or dry-run. +- **`skiff domains remove` doesn't shrink the shared certificate's SAN.** + A removed hostname may still be listed on the app's certificate until + the next `add` re-syncs the domain set. Harmless, since nothing routes + to it anymore. --- @@ -468,6 +602,9 @@ Carried over from the project's own README, plus what testing surfaced: | `git push` says `Everything up-to-date` but you wanted to retry a failed deploy | The git ref already advanced despite the deploy failing - nothing new to push | New commit (`git commit --allow-empty`), or on the host: `skiff deploy --app --new --ref refs/heads/main` | | `build: \`make\` did not produce ./app` | Your Makefile's default target ran (or was skipped as "up to date") without ultimately leaving a file named `app` in the repo root | Run `make -B` locally in a clean checkout and confirm `./app` exists after; check the target actually named `app` (or a dependency of the default target) writes there | | Health check fails and you're not sure why | - | `jexec sockstat -4 -l` to check for a listener; `jexec ps aux` to see if the process is even running; try `jexec /app/current/app` directly to see startup errors that daemon(8) would otherwise swallow | +| Build fails inside the jail with a missing command (`go: not found`, etc.) even though it's installed on the host | The base jail image itself is missing it, or FreeBSD installed it under a versioned name (`go125`, not `go`) | `jexec which `; if it's there under a versioned name, symlink a default into the base image's `/usr/local/bin`, re-snapshot, and `skiff rebuild --app --base-snapshot ` | +| Build fails inside the jail on any network call (module download, `pkg install`), or `pkg`/`fetch`/`go mod` just time out | The app's `--ip` doesn't actually route from this host - classic jails share the host's routing table, but an out-of-subnet address (e.g. `10.0.0.5` on a host whose real subnet is `192.168.1.0/24`) can send packets that silently never get a response behind some NAT/bridged setups | `jexec fetch -T5 -o /dev/null https://` to confirm; if it hangs, re-provision with an `--ip` in the same subnet as the host's own address on `--iface` (see [Provisioning a new app](#provisioning-a-new-app)) | +| `skiff domains add` fails with a certbot/ACME error | Most likely DNS isn't pointed at the app's `--ip` yet; less commonly, the app's `--ip` doesn't route outbound from this host (see above) | Fix DNS, then re-run `skiff domains add` with the same hostname - safe to retry, the app stays on plain HTTP in the meantime and nothing is left half-configured | --- @@ -475,7 +612,7 @@ Carried over from the project's own README, plus what testing surfaced: ``` skiff new-app --name --ip [--port 8080] [--health-path /healthz] - [--server-name ] [--base-snapshot ready] [--iface vtnet0] + [--base-snapshot ready] [--iface vtnet0] skiff deploy --app --new [--ref refs/heads/main] [--old ] (normally invoked by the post-receive hook, not by hand) @@ -489,4 +626,8 @@ skiff rebuild --all [--base-snapshot ready] [--iface vtnet0] skiff config set --app KEY=VALUE [KEY=VALUE...] skiff config unset --app KEY [KEY...] skiff config list --app + +skiff domains add --app [--primary] +skiff domains remove --app +skiff domains list --app ``` -- cgit v1.3