How an upgrade works
One external installer, one journaled transaction, one receipt, and a defined answer for every way it can go wrong.
Where K stops
The machine is remote and nobody operates it. The upgrade has to stop the running service, start the new one, and keep working while the service is down; if the new version does not start, the old one has to come back on its own, because nobody can log in to put it back. K's answer is one journaled transaction with a defined result, run by a process that is not the application. This page is about that transaction.
What K does not decide is everything around it: who may ask, whether anyone is present to consent, and what is already on the machine. K's transaction covers a machine it already manages and one whose upgrade was interrupted; the installer built on K decides the rest. That behavior is the installer contract, a separate and advisory page.
The pieces
One external installer, launched from every entry, owns the transaction and outlives the application it replaces. It has a small number of parts.
- Bootstrap
- Your
install.sh, or theself upgradecommand inside your application. Downloads and verifies the installer, launches it, relays the exit code. Contains no upgrade logic. - Installer
- What you ship, in two parts. The supervisor is a temporary process that acquires and verifies the runner, executes it and, if it dies, runs bounded recovery. The runner is K plus your adapter bundled into one executable; it serves one request and exits. A running runner is a worker.
- Adapter
- Your trusted code, fixed at build time: where releases come from, whether an upgrade may proceed, how to reach the controller.
- Controller
- Your program that actually stops, starts and probes the application. The adapter calls it directly or through
createCommandHost. - State directory
- The lock, the journal, two slots and the receipts. Recovery needs this state, a compatible runner, its runtime and the controller. It does not need release distribution.
The slots are stable and experiment. They are positions on disk, not release channels; a product channel also called "stable" is unrelated.
One upgrade, start to finish
- bootstrap Asks the supervisor for one operation: an id and a target version. The id is what everything else binds to.
- supervisor Downloads the runner, checks its hash and size, writes it to scratch space and runs it with the request on stdin.
- worker Takes the installation lock. An unfinished earlier operation is settled first. The same id replays its result; a new id may proceed only after recovery succeeds. controller is asked to fence first: to confirm that nothing an earlier worker queued can still land.
- adapter Asks your release source for exactly the target version. worker verifies the bytes and writes them into the experiment slot. Nothing running has changed yet.
journal: staged - controller Quiesces, parking in-flight work, then stops the current service and confirms the old process is gone.
journal: handing-over, with the pre-stop startId - controller Starts the candidate from the experiment slot. worker probes it. The probe must come from one live process and report the expected version, a pid and a startId. If the startId journaled before the stop comes back now, the old process was never replaced and the upgrade rolls back.
journal: running-experiment, readback - worker Writes promote intent. Before this line, the safe move is always to put the old version back; after it, the safe move is always to finish the promotion.
journal: promote intent - worker Experiment becomes stable. controller resumes parked work. The receipt is persisted in
operation.jsonand archived before any later operation begins. The worker prints its response and exits 0; the supervisor relays it.
When something goes wrong
The candidate is bad
It does not start, it reports the wrong version, or it never passes the probe. The worker stops it, starts stable again, resumes parked work, and records rolled-back. Exit code 1. The stable bytes were never touched. This is the routine failure and the one K is built around.
The worker dies
A crash, a kill, or a budget timeout. The supervisor:
- terminates a timed-out worker and waits for its observed exit; a deadline alone does not permit takeover;
- starts a recovery worker bound to the same operation id;
- that worker takes the lock, checks the operation identity, and asks the controller to fence before replaying lifecycle effects, so actions queued by the earlier worker cannot land later. An already terminal operation simply replays its receipt.
Recovery needs no network and does not guess. It reads the journal and does one of two things. Recovery never starts a new upgrade, even if the original request was one.
Attempts and elapsed time are bounded. If recovery does not settle within them, the supervisor exits 3, leaves the verified runner and a recovery.json in scratch space, and prints where. resumeRunner on that file retries the same recovery offline.
Everything dies
Power loss, reboot, or the whole invocation killed. The state directory survives, and the next installer invocation settles the unfinished operation before it accepts new work. K does not install a watchdog, so something has to run the installer again: your product's OS startup hook, or a person.
Beyond recovery
Recovery settles K's own work. A machine it cannot settle, or whose records it cannot read, is broken, and there K refuses rather than guesses. The installer then reinstalls it in the same run, keeping the old state aside; what that looks like is in the installer contract, and what it must keep is the repair boundary.
Reading the result
Every run prints one JSON response on stdout and returns an exit code. Inspect both.
| Exit | Meaning | What to do |
|---|---|---|
| 0 | Promoted or up-to-date; status readable; recovery that found nothing to settle | Check the action and receipt |
| 1 | Rolled back, or failed before any change | Read operation.operation.outcome and the reason |
| 2 | Held: not consented, owned elsewhere, incompatible, or no exact version | Nothing changed; a new attempt needs a new id |
| 3 | Unresolved | K's honest answer, never the user's: an installer built on K reinstalls; a caller of K alone keeps the recovery file |
Two things surprise people. A recover that successfully restored stable after a bad candidate exits 1, because the recorded outcome is a rolled-back upgrade. That is the honest answer. And status reads the last receipt, not the live service; it can say genesis on a machine that is running fine.
Response shapes and exit codes are in the reference. What each entry owes the caller for each of these outcomes is in the installer contract.