Shipping

hearth ship is the entry point. Everything downstream is GitHub-driven, not laptop-driven. This page is the canonical reference for how shipping works across every KATforge service.

Quick start

hearth ship api                          # → prod, tail CI run
hearth ship api --qa                     # → qa namespace (release marked prerelease)
hearth ship api --draft                  # build only, don't deploy
hearth promote api v0.0.28-rc.1          # promote a qa prerelease to prod
hearth ship                              # ships every service in SHIP_ALL

What happens in 10 seconds:

hearth ship api
   ├─ pre-ship tests run locally   (~5 min for api, instant for frontends)
   ├─ imp ship --patch --stable    (commit + version bump + tag + push)
   ├─ POST /releases               (creates GitHub Release)
   └─ tail the CI run              (step transitions stream into your terminal)
                                                       │
                                                       ▼   release.published
                                              GitHub Actions kicks off

You walk away when CI prints success. No alt-tabs.

The pipeline

Every shippable repo has the same shape: a local pre-ship gate, a GitHub Release, a workflow that fires on release.published.

                  your laptop                                GitHub                            k3s
                  ───────────                                ──────                            ───
hearth ship api
   pre-ship tests ──────────────────────────────────────────────────────────────►   pass / fail (blocks)
   imp ship --patch ─────────────────────────► refs/tags/v0.0.28 (push)
   release.create  ─────────────────────────► Release(tag=v0.0.28, latest)
                                                       │
                                                       ▼
                                  .github/workflows/release.yml fires
                                                       │  (calls katforge/hearth template)
                                  ┌────────────────────┴────────────────────┐
                                  ▼                                          ▼
                          docker build                              azure/setup-kubectl
                          docker push  ─────────► ghcr.io/katforge/api:v0.0.28
                                                                              │
                                                                              ▼
                                                                kubectl set image -n default ──►   pod restart
                                                                kubectl rollout status                init: migrate
                                                                                                     container: v0.0.28
   ◄─── tail run via GitHub API ─── (status of each step streamed back)

Five clean responsibilities:

Stage routing via the prerelease flag

The same image can deploy to qa or default based on a single boolean on the Release.

Promoting QA to prod (no rebuild)

hearth promote api v0.0.28-rc.1

Edits the release: prerelease=false, make_latest=true. The same release.published event fires, but with prerelease=false now. CI deploys the same image to default. Zero rebuild.

This is the cleanest "promote what you tested" flow possible without GitOps. The hearth promote command is just a thin wrapper around the GitHub API edit; the underlying mechanism is fully GitHub-native.

Service registry: hearth/services.yaml

Every shippable service lives in hearth/services.yaml:

api:
   directory:    api.katforge.com
   image:        ghcr.io/katforge/api.katforge.com
   dockerfile:   Dockerfile.prod
   k8s_name:     api
   dev_host:     api.dev.katforge.com
   pre_ship_cmd: ./bin/phpunit --testsuite Feature

geargoblins:
   directory:        geargoblins.com/ui
   image:            ghcr.io/katforge/geargoblins.com
   k8s_name:         geargoblins
   shippable:        false
   shippable_reason: No Dockerfile in geargoblins.com/ui yet.

To add a service: add a block to services.yaml. To exclude one from hearth ship all: set shippable: false. No Python edits.

All projects, same workflow

Every shippable repo has .github/workflows/release.yml that calls a single reusable template at katforge/hearth/.github/workflows/release-template.yml. The caller stub is ~12 lines:

# api.katforge.com/.github/workflows/release.yml
on:
   release:
      types: [published]

jobs:
   ship:
      uses: katforge/hearth/.github/workflows/release-template.yml@main
      with:
         service:    api
         image:      ghcr.io/katforge/api.katforge.com
         deployment: api
         dockerfile: Dockerfile.prod
      secrets:
         KATFORGE_GITHUB_TOKEN: ${{ secrets.KATFORGE_GITHUB_TOKEN }}
         K3S_KUBECONFIG: ${{ secrets.K3S_KUBECONFIG }}

The template handles checkout, optional sibling multi-checkout, optional npmrc generation, optional monorepo manifest pull, docker build/push, kubectl set image, and rollout. Drift between repos becomes structurally impossible — the only per-repo state is the inputs.

What inputs each project passes

Inputs reflect each project's existing build setup. The mechanism is uniform.

Migrations

Schema changes ship through a Doctrine migrate init container declared in hearth/k8s/api/deployment.yaml. The init image is pinned to the same tag as the main container, so the code that authored the migrations is the code that runs them — no risk of a newer migration applying against an older codebase. A failed migration fails the rollout, so kubectl rollout status is the migration health check too. Bad schema → new ReplicaSet stuck pending → old pod keeps serving traffic.

Pre-ship test gate

Pre-ship tests run on your laptop, before any git mutation. Configured per service in hearth/services.yaml:

api:
   pre_ship_cmd: ./bin/phpunit --testsuite Feature

The frontends have no pre-ship gate yet. Add one if you want by setting pre_ship_cmd in services.yaml.

If you need to ship past a failing or hung test (production hotfix), --skip-tests bypasses the gate. The gate is local-only — CI doesn't re-run it. Trust your tests.

CI watch (default behavior)

hearth ship tails the GitHub Actions run by default. Each step transitions print to your terminal as they happen — final summary looks like:

✓ api v0.0.28 → release published as latest (prod)
✓ Build and push image
✓ Deploy to k3s
✓ api v0.0.28 deployed

Each ▸ step prints when it begins and the matching ✓ lands when it completes, so you see progress in real time without alt-tabbing to GitHub. Use --no-watch to skip the tail and exit immediately after the release POST.

First-time CI setup

hearth setup --ci

This bootstraps the entire CI plumbing in one command:

1. Applies hearth/k8s/ci/rbac.yaml (creates ci namespace, github-deployer SA, scoped RBAC, long-lived token Secret).
2. Reads the SA token + cluster CA + server URL, builds a kubeconfig, base64-encodes it.
3. Sets two repo secrets across every shippable repo: K3S_KUBECONFIG (the kubeconfig) and KATFORGE_GITHUB_TOKEN (the value of your laptop's GITHUB_KATFORGE_TOKEN, used for ghcr push and sibling repo checkouts). The secret name flips word order because GitHub reserves the GITHUB_ prefix for stored secrets.

After this completes, hearth ship works end-to-end. Re-run when adding a new shippable service or rotating the kubeconfig.

Required tooling

hearth ship needs three tools on your laptop:

Hearth no longer requires docker or kubectl for shipping. They stay required for hearth status, hearth logs, and hearth rollback.

Failure modes

The pipeline has natural checkpoints; failure at each one stops cleanly without partial state.

The migration init container is the load-bearing safety net. A bad schema change fails the rollout, the new ReplicaSet never goes ready, the old pod keeps serving traffic. No manual rollback needed for the half-deployed case.

Manual deploys

Sometimes you need to redeploy without changing code. Examples: a config-only secret rotation, a botched first deploy, a workflow that crashed mid-flight.

hearth ship api --no-version    # POST a Release for the current tag, no version bump

This skips imp ship entirely and creates a Release for whatever the most recent tag is. CI fires as if nothing else happened.

If the Release already exists, the POST will 422. Either delete it first in the UI or bump the version with a regular hearth ship.

Rollback

Two paths.

hearth rollback api                # fast: kubectl rollout undo

Reverts the cluster, leaves the broken Release marked latest on GitHub. Fine if you'll re-ship a fix soon.

For a clean record: in the GitHub Releases UI, edit the broken release, mark it as draft (or delete), then "Set as latest" on the previous good release. The workflow fires again with the older tag, deploying the same image to k3s. Cluster and "what's in prod on GitHub" stay in sync.

Appendix: idiomatic vs. pragmatic

The pipeline is well-engineered for a single-instance k3s with a small team. A few deliberate non-best-practices worth knowing about:

What still uses local docker

hearth up, hearth dev, and hearth status still use your local docker daemon and kubectl. Only hearth ship is CI-driven now.