path: root/README.md

# Skiff

A minimal PaaS for FreeBSD + Nginx apps, in any language with a Makefile.
`git push` triggers a build, an optional app-defined release phase, a ZFS
snapshot, a cutover, and a health check - with automatic rollback to the
pre-deploy snapshot if the new release fails its health check. One binary,
no daemon, no separate metadata database - jail.conf, ZFS datasets, and a
couple of flat files per app are the only state.

Skiff has no opinion on your language or runtime - the only thing it
requires of an app is a `Makefile` that produces an executable named `app`,
and that `app` accepts TCP traffic on an assigned port. See [What an app
needs to provide](#what-an-app-needs-to-provide).

## Requirements

- FreeBSD 15, ZFS root
- `nginx` and `git` - neither ships in base; install with
  `pkg install nginx git-lite`
- `certbot` - only needed if you'll route real domains via `skiff domains`;
  install with `pkg install py312-certbot` (package name tracks the default
  Python version, so this may drift)
- `jail_enable="YES"` in `/etc/rc.conf` (`sysrc jail_enable=YES`) - `service
  jail start`/`restart` refuse to run without it
- Whatever toolchain your apps' Makefiles invoke to build - `go`, `python`,
  `rustc`, `node`, etc. - baked into the base jail image (see [One-time host
  setup](#one-time-host-setup)); builds run inside the jail via `jexec`, not
  on the host itself. `make` itself is already in FreeBSD base (bmake), so
  it's the one build tool Skiff never asks you to install. Watch for
  versioned port names (`go125`, `python312`, ...) - FreeBSD installs the
  literal versioned binary, not a bare `go`/`python3`. Check for an
  unversioned meta-port first (`go` itself is one - see [ZFS
  layout](#zfs-layout)); only symlink a default by hand if no meta-port
  exists for what you need.
- The build step needs real outbound network access from inside the jail
  (module/package downloads) - see the `--ip` note under [First
  deploy](#first-deploy).

## One-time host setup

### ZFS layout

```
# zfs create zroot/skiff
# zfs create -o mountpoint=/skiff/base zroot/skiff/base
# bsdinstall jail /skiff/base
```

`bsdinstall jail` only lays down the bare base system - it ships `pkg(8)`'s
bootstrap stub but never actually runs it, so a jail cloned from this alone
has no working `pkg` of its own (files can still be `pkg -c`-installed into
it from the host, but nothing *inside* the jail can manage them, and
`jexec <app> pkg ...` fails with "the package management tool is not yet
installed"). Bootstrap pkg into the image itself, before snapshotting:

```
# mount -t devfs devfs /skiff/base/dev
# env ASSUME_ALWAYS_YES=yes pkg -c /skiff/base bootstrap -f
```

(`-f` because the stub already counts as "pkg is present" otherwise; devfs
is needed for pkg's own network/TLS calls inside a bare chroot - `fetch(1)`
wants `/dev/null` and `/dev/random`.) Now install whatever your apps'
Makefiles actually need to build:

```
# pkg -c /skiff/base install -y go
```

FreeBSD's versioned language ports (`go125`, `python312`, ...) install as
that literal binary name, not a bare `go`/`python3` - a Makefile invoking
the unversioned command needs one. Check for an unversioned meta-port
first: `go` is exactly that - it depends on whichever `goNNN` is current
and installs the `go`/`gofmt` symlinks itself (`pkg install -y go`, not
`go125` directly, is the whole fix). If no meta-port exists for what you
need, symlink a default into `/usr/local/bin` by hand instead.

Unmount devfs and snapshot once the image can actually build and run apps:

```
# umount /skiff/base/dev
# zfs snapshot zroot/skiff/base@default
# zfs create -o mountpoint=none zroot/skiff/appbase
# zfs create -o mountpoint=none zroot/skiff/appdata
# mkdir -p /skiff/apps
```

`default` is the snapshot every app's dataset gets cloned from unless
`--base-snapshot` says otherwise. Re-snapshot
under a new tag after patching the base (e.g. `security-2026-08`) and pass
`--base-snapshot` to `new-app` to pick it up.

`zroot/skiff/appbase` and `zroot/skiff/appdata` must each exist as their own
dataset, not a plain directory - `new-app` clones/creates straight into them
(`zroot/skiff/appbase/<name>`, `zroot/skiff/appdata/<name>`), and `zfs
clone`/`zfs create` require the parent dataset to already exist. Their own
mountpoint doesn't matter (`none` is fine) since every per-app dataset sets
its own explicit mountpoint under `/skiff/apps/<name>` - `appbase/<name>` at
`/skiff/apps/<name>` itself (the jail's root), `appdata/<name>` at
`/skiff/apps/<name>/app`. `/skiff/apps` itself is a plain directory, like
`/skiff/repos` - nothing needs to be a dataset at that exact path.

Keeping the OS layer (`appbase`) and the app's persistent data (`appdata`)
on separate datasets is what lets `skiff rebuild` destroy and reclone the
former without any risk to the latter.

### Repos directory

```
# mkdir -p /skiff/repos
```

### Jail config

FreeBSD doesn't ship an `/etc/jail.conf`, and at boot `service jail start`
(no jail named) only creates jails it finds in that one file - it never
looks at `jail.conf.d/` on its own. Point it there once:

```
# echo '.include "/etc/jail.conf.d/*.conf";' > /etc/jail.conf
```

Without this, `skiff new-app` and manual `service jail start <app>` still
work fine - both read `jail.conf.d/<app>.conf` directly - so the gap stays
hidden until the host reboots and no app jail comes back up.

### Nginx

Add once to `/usr/local/etc/nginx/nginx.conf`, inside the `http` block:

```
include /usr/local/etc/nginx/skiff.d/*.conf;
```

```
# mkdir -p /usr/local/etc/nginx/skiff.d
```

Skiff only ever writes files under `skiff.d/` and reloads - it never touches
`nginx.conf` itself.

### TLS (Let's Encrypt)

Skiff issues certificates itself via `certbot certonly` (webroot validation,
never certbot's own `--nginx` plugin - Skiff owns `skiff.d/*.conf` and would
fight any other writer of it). Register the certbot account once:

```
# certbot register --agree-tos -m you@example.com
```

Renewal rides on FreeBSD's own `periodic(8)`, same as any other certbot
install - Skiff runs no daemon or scheduler of its own for this:

```
# sysrc -f /etc/periodic.conf weekly_certbot_enable="YES"
```

`certonly` mode doesn't reload nginx after a renewal the way certbot's
installer plugins do, so add that as a deploy hook once:

```
# mkdir -p /usr/local/etc/letsencrypt/renewal-hooks/deploy
# printf '#!/bin/sh\nnginx -s reload\n' > /usr/local/etc/letsencrypt/renewal-hooks/deploy/reload-nginx
# chmod +x /usr/local/etc/letsencrypt/renewal-hooks/deploy/reload-nginx
```

### Install skiff

```
$ GOOS=freebsd GOARCH=amd64 go build -o skiff ./cmd/skiff
# install -m 755 skiff /usr/local/sbin/skiff
```

## Deploying an app

### First deploy

```
# skiff new-app --name myapp --ip 192.168.1.50 --port 8080
```

Clones the base dataset, writes `/etc/jail.conf.d/myapp.conf` and the jail's
own rc.d service, creates the bare repo at `/skiff/repos/myapp.git` with its
`post-receive` hook installed, and starts the (empty) jail. Prints the
remote URL to add locally.

`--ip` must actually route on the host's `--iface` interface - in practice,
an address in the same subnet as the host's own primary address on that
interface, not an arbitrary block like `10.0.0.0/8`. Classic (non-VNET)
jails share the host's routing table, so an out-of-subnet address can still
send packets - they just may never get a response, silently, if whatever
sits upstream (a NAT gateway, a virtualized bridge) only answers its own
subnet. This didn't matter before builds moved inside the jail; now every
deploy's build step needs real outbound network access from that exact
address (module downloads, `pkg install`, etc).

### Domains & TLS

`new-app` only provisions the jail - nothing is routed until you add a
domain:

```
# skiff domains add --app myapp --primary myapp.example.com
# skiff domains add --app myapp www.myapp.example.com
# skiff domains list --app myapp
# skiff domains remove --app myapp www.myapp.example.com
```

`add` routes the hostname over plain HTTP immediately, then requests or
expands a single certificate covering every domain currently registered for
the app (named after whichever is primary). If DNS isn't pointed at the
app's `--ip` yet, issuance fails and the command exits non-zero - the app
stays reachable over HTTP, and `add` is safe to re-run once DNS is fixed,
same as retrying a failed deploy. The first domain added becomes primary
automatically; every other domain 301-redirects to it, over HTTP and HTTPS
alike. `remove` only updates routing - it never calls out to certbot, so a
DNS or network problem can't make it fail.

### Every deploy after

```
$ git remote add skiff root@host:/skiff/repos/myapp.git
$ git push skiff main
```

The push blocks until the deploy finishes - success or failure prints
straight to your terminal, no dashboard to check.

### Rollback

```
# skiff rollback --app myapp --list
# skiff rollback --app myapp --to pre-a1b2c3d
```

Rolls the app's entire ZFS dataset back to that snapshot (code and any
release-phase side effects together) and restarts the jail.

This uses `zfs rollback -r`, which **destroys every snapshot newer than the
target** - ZFS can't roll back past one otherwise. Rolling back to an old
snapshot permanently discards the ability to return to anything deployed
in between; there's no separate "undo the rollback."

### Rebuilding a jail

```
# skiff rebuild --app myapp
# skiff rebuild --all
```

Discards the app's OS layer (`/etc`, `/usr`, `/bin`, `jail.conf`, the jail's
rc.d script, `dma.conf`) and reclones it fresh from the base image, leaving
`/app` (releases, current, env, `meta.json`, `data.db`) completely
untouched - the two live on separate ZFS datasets. Use this to pick up a
patched/upgraded base image (`--base-snapshot`) on an app that already
exists, or to regenerate stale jail config after a Skiff upgrade, without
touching the app's own data. Guarded exactly like a deploy: the old OS
layer is kept until the rebuilt jail passes its health check, and swapped
back automatically if it doesn't.

## What an app needs to provide

A `Makefile` in the repo root, and nothing else Skiff cares about - no
required language, framework, or file layout beyond this contract:

- Running bare `make` (Skiff runs `make -B`, so it never trusts a stale
  build artifact left in the checkout; every deploy is a fresh worktree
  anyway) must leave an executable file named `app` in the repo root.
  What produces it is entirely up to you - `go build -o app .`, a
  `cp target/release/myapp app` after `cargo build --release`, a shell
  script with a `#!/usr/bin/env python3` shebang that `chmod +x`'d itself
  into place, whatever. Skiff execs this file directly (via `daemon(8)`),
  and the kernel doesn't care whether it's a native binary or a
  script - only that it's marked executable and has a valid interpreter
  line if it's not.
- That executable listens on the port given to `new-app` (bind the
  wildcard address, not `127.0.0.1`), and answers the configured health
  path with a non-3xx status once ready.
- Optionally, a `release` target in the same Makefile - run once inside
  the jail *before* cutover (against the new release's own checkout, not
  whatever `current` still points at), gating the deploy on its exit
  code. Skiff doesn't interpret this target; it's entirely the app's own
  convention (`sqlite3` migrations, cache warming, whatever). Detected by
  asking `make` itself via `make -n release`, not by parsing the
  Makefile - it can be defined however you like (directly, via an
  include, a pattern rule).

```makefile
.PHONY: all release
all: app

app: main.go go.mod
	go build -o app .

release:
	sqlite3 /app/data.db < migrations/0004_add_x.sql
```

See [USER_GUIDE.md](USER_GUIDE.md) for a full walkthrough with a real
example app.

Runtime dependencies (a Python interpreter, the Node runtime, etc.) must
be present in the jail's base image if your `app` isn't a self-contained
static binary - that's a base-jail provisioning concern, not something
the Makefile handles.

## Known gaps

- No IP allocation - `--ip` is operator-chosen. Fine for a handful of apps
  on one host; would want a small pool allocator beyond that.
- Old `releases/<sha>` directories are never pruned. The ZFS snapshot is the
  real rollback mechanism; the release dirs are just there for the instant
  symlink flip and are safe to clean up by hand.
- Single host only. Scaling to a second host is a `zfs send`/`receive` of an
  app's dataset plus the same jail.conf/nginx steps - not yet automated here.
- `skiff domains remove` doesn't shrink the shared certificate's SAN - a
  removed hostname may still be listed on it until the next `add` re-syncs
  the domain set. Harmless, since nothing routes to it anymore.