diff options
Diffstat (limited to 'USER_GUIDE.md')
| -rw-r--r-- | USER_GUIDE.md | 149 |
1 files changed, 145 insertions, 4 deletions
diff --git a/USER_GUIDE.md b/USER_GUIDE.md index fe78bd0..63d6517 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -224,12 +224,23 @@ this in a hurry - it's more destructive than "undo." ``` # pkg install nginx git-lite ``` +- `certbot` - only needed if you'll route real domains via `skiff domains`: + ``` + # pkg install py312-certbot + ``` + (the package name tracks the default Python version, so this may drift) - Whatever your apps' Makefiles actually invoke to build - `go`, `python3`, `rustc`, `node`, etc. - baked into the base jail image (see [ZFS layout](#1-zfs-layout) below); builds run *inside* the jail via `jexec`, not on the host. Skiff itself doesn't care which; `make` is already in FreeBSD base (bmake), so it's never something you need - to install separately. + to install separately. Watch for versioned port names - FreeBSD's + `go125`, `python312`, etc. install as that literal binary, not a bare + `go`/`python3`; symlink a default into the base image's + `/usr/local/bin` if a Makefile invokes the unversioned name. The build + step also needs real outbound network access from inside the jail + (module/package downloads) - see the `--ip` note under [Provisioning a + new app](#provisioning-a-new-app). - `jail_enable="YES"` in `/etc/rc.conf`: ``` # sysrc jail_enable=YES @@ -246,6 +257,42 @@ this in a hurry - it's more destructive than "undo." # zfs create zroot/skiff # zfs create -o mountpoint=/skiff/base zroot/skiff/base # bsdinstall jail /skiff/base +``` + +`bsdinstall jail` only lays down the bare base system - it ships `pkg(8)`'s +bootstrap stub but never actually runs it, so a jail cloned from this alone +has no working `pkg` of its own (files can still be `pkg -c`-installed into +it from the host, but nothing *inside* the jail can manage them, and +`jexec <app> pkg ...` fails with "the package management tool is not yet +installed"). Bootstrap pkg into the image itself, before snapshotting: + +``` +# mount -t devfs devfs /skiff/base/dev +# env ASSUME_ALWAYS_YES=yes pkg -c /skiff/base bootstrap -f +``` + +(`-f` because the stub already counts as "pkg is present" otherwise; devfs +is needed for pkg's own network/TLS calls inside a bare chroot - `fetch(1)` +wants `/dev/null` and `/dev/random`.) Now install whatever your apps' +Makefiles actually need to build: + +``` +# pkg -c /skiff/base install -y go125 +``` + +Watch for versioned port names - FreeBSD installs `go125` as literally +`go125`, not `go`, so a Makefile invoking the bare command needs a default +symlinked in: + +``` +# ln -sf go125 /skiff/base/usr/local/bin/go +# ln -sf gofmt125 /skiff/base/usr/local/bin/gofmt +``` + +Unmount devfs and snapshot once the image can actually build and run apps: + +``` +# umount /skiff/base/dev # zfs snapshot zroot/skiff/base@ready # zfs create -o mountpoint=none zroot/skiff/appbase # zfs create -o mountpoint=none zroot/skiff/appdata @@ -337,7 +384,38 @@ include /usr/local/etc/nginx/skiff.d/*.conf; Skiff only ever writes files under `skiff.d/` and reloads - it never touches `nginx.conf` itself. -#### 5. Install skiff +#### 5. TLS (Let's Encrypt) + +Skiff issues certificates itself via `certbot certonly` (webroot +validation, never certbot's own `--nginx` plugin - Skiff owns +`skiff.d/*.conf` and regenerates it on every domain/deploy change, and +would fight any other writer of that file). Register the certbot account +once: + +``` +# certbot register --agree-tos -m you@example.com +``` + +Renewal rides on FreeBSD's own `periodic(8)`, the same as any other +certbot install - Skiff runs no daemon or scheduler of its own for this, +and `certbot renew` is host-wide by design, so one cron job covers every +app's certificates regardless of when each was issued: + +``` +# sysrc -f /etc/periodic.conf weekly_certbot_enable="YES" +``` + +`certonly` mode doesn't reload nginx after a renewal the way certbot's +installer plugins do, so add that as a deploy hook once - certbot runs +every script under this directory after any successful renewal: + +``` +# mkdir -p /usr/local/etc/letsencrypt/renewal-hooks/deploy +# printf '#!/bin/sh\nnginx -s reload\n' > /usr/local/etc/letsencrypt/renewal-hooks/deploy/reload-nginx +# chmod +x /usr/local/etc/letsencrypt/renewal-hooks/deploy/reload-nginx +``` + +#### 6. Install skiff Cross-compile from any machine with Go, then copy the binary over: @@ -350,7 +428,7 @@ $ scp skiff root@host:/tmp/skiff ### Provisioning a new app ``` -# skiff new-app --name myapp --ip 10.0.0.5 --port 8080 --server-name myapp.example.com +# skiff new-app --name myapp --ip 192.168.1.50 --port 8080 ``` This clones the base dataset, writes `/etc/jail.conf.d/myapp.conf` and @@ -374,6 +452,58 @@ from the base image, so mail sent from inside the jail (e.g. by cron or the app itself) is attributed to `myapp.skiff.local` instead of the base image's own hostname. +**`--ip` must actually route on the host's `--iface` interface** - in +practice, an address in the same subnet as the host's own primary +address on that interface, not an arbitrary block like `10.0.0.0/8`. +Classic (non-VNET) jails share the host's routing table, so an +out-of-subnet address can still send packets - they just may never get a +response, silently, if whatever sits upstream (a NAT gateway, a +virtualized bridge) only answers its own subnet. This didn't matter +before builds moved inside the jail (see [Requirements](#requirements)); +now every deploy's build step needs real outbound network access from +that exact address (module downloads, `pkg install`, etc), not just +inbound reachability for the app itself. + +### Domains and TLS + +`new-app` only provisions the jail - nothing is routed until you add a +domain: + +``` +# skiff domains add --app myapp --primary myapp.example.com +# skiff domains add --app myapp www.myapp.example.com +# skiff domains list --app myapp +# skiff domains remove --app myapp www.myapp.example.com +``` + +`add` writes the hostname to the app's `meta.json`, routes it over plain +HTTP immediately (so the ACME HTTP-01 challenge below has somewhere to +land), then requests or expands a single certificate covering every +domain currently registered for the app, named after whichever is +primary - matching how a real multi-domain vhost is normally issued (one +SAN cert), not a cert per hostname. + +If DNS isn't pointed at the app's `--ip` yet, certificate issuance fails +and the command exits non-zero - the app stays reachable over HTTP, and +`add` is safe to re-run once DNS is fixed, exactly like retrying a failed +deploy (see [If the health check +fails](#if-the-health-check-fails)). Nothing about a failed `add` is +destructive. + +The first domain added for an app becomes primary automatically; +`--primary` reassigns it explicitly. Every non-primary domain +301-redirects to the primary, over HTTP and HTTPS alike, once its own +hostname is covered by the certificate. + +`remove` only updates routing - unlike `add`, it never calls out to +certbot, so a DNS or network problem can't make it fail. It's refused if +you try to remove the current primary while other domains remain - +reassign primary first (`skiff domains add --primary <other-hostname>`). + +Domain/TLS changes never restart the jail and never run a health check - +unlike a code deploy or `skiff config`, they don't touch the running app +at all, only nginx and the certificate. + ### Rollback ``` @@ -452,6 +582,10 @@ Carried over from the project's own README, plus what testing surfaced: steps - not automated. - **`skiff rollback -r` is destructive** (see above) - there is currently no confirmation prompt or dry-run. +- **`skiff domains remove` doesn't shrink the shared certificate's SAN.** + A removed hostname may still be listed on the app's certificate until + the next `add` re-syncs the domain set. Harmless, since nothing routes + to it anymore. --- @@ -468,6 +602,9 @@ Carried over from the project's own README, plus what testing surfaced: | `git push` says `Everything up-to-date` but you wanted to retry a failed deploy | The git ref already advanced despite the deploy failing - nothing new to push | New commit (`git commit --allow-empty`), or on the host: `skiff deploy --app <app> --new <sha> --ref refs/heads/main` | | `build: \`make\` did not produce ./app` | Your Makefile's default target ran (or was skipped as "up to date") without ultimately leaving a file named `app` in the repo root | Run `make -B` locally in a clean checkout and confirm `./app` exists after; check the target actually named `app` (or a dependency of the default target) writes there | | Health check fails and you're not sure why | - | `jexec <app> sockstat -4 -l` to check for a listener; `jexec <app> ps aux` to see if the process is even running; try `jexec <app> /app/current/app` directly to see startup errors that daemon(8) would otherwise swallow | +| Build fails inside the jail with a missing command (`go: not found`, etc.) even though it's installed on the host | The base jail image itself is missing it, or FreeBSD installed it under a versioned name (`go125`, not `go`) | `jexec <app> which <cmd>`; if it's there under a versioned name, symlink a default into the base image's `/usr/local/bin`, re-snapshot, and `skiff rebuild --app <app> --base-snapshot <new-tag>` | +| Build fails inside the jail on any network call (module download, `pkg install`), or `pkg`/`fetch`/`go mod` just time out | The app's `--ip` doesn't actually route from this host - classic jails share the host's routing table, but an out-of-subnet address (e.g. `10.0.0.5` on a host whose real subnet is `192.168.1.0/24`) can send packets that silently never get a response behind some NAT/bridged setups | `jexec <app> fetch -T5 -o /dev/null https://<some-public-ip>` to confirm; if it hangs, re-provision with an `--ip` in the same subnet as the host's own address on `--iface` (see [Provisioning a new app](#provisioning-a-new-app)) | +| `skiff domains add` fails with a certbot/ACME error | Most likely DNS isn't pointed at the app's `--ip` yet; less commonly, the app's `--ip` doesn't route outbound from this host (see above) | Fix DNS, then re-run `skiff domains add` with the same hostname - safe to retry, the app stays on plain HTTP in the meantime and nothing is left half-configured | --- @@ -475,7 +612,7 @@ Carried over from the project's own README, plus what testing surfaced: ``` skiff new-app --name <app> --ip <ip> [--port 8080] [--health-path /healthz] - [--server-name <name>] [--base-snapshot ready] [--iface vtnet0] + [--base-snapshot ready] [--iface vtnet0] skiff deploy --app <app> --new <sha> [--ref refs/heads/main] [--old <sha>] (normally invoked by the post-receive hook, not by hand) @@ -489,4 +626,8 @@ skiff rebuild --all [--base-snapshot ready] [--iface vtnet0] skiff config set --app <app> KEY=VALUE [KEY=VALUE...] skiff config unset --app <app> KEY [KEY...] skiff config list --app <app> + +skiff domains add --app <app> [--primary] <hostname> +skiff domains remove --app <app> <hostname> +skiff domains list --app <app> ``` |