ADR 0005: A single-replica API, SQLite, and what that costs
Date: 2026-09-11 Status: accepted
Context
The API holds the audit log, accounts, sessions, licence state and terminal recordings. The obvious instinct for a product sold to banks is PostgreSQL and three replicas behind a service. That instinct is wrong here, and it is worth writing down why, because it is the first question a customer's architect asks.
Nazeel is installed by the customer, into their cluster, often air-gapped, frequently by someone who is evaluating it on a Friday afternoon. Every dependency is something they must provision, back up, patch and explain to their own DBA team before they can see the product work. A required PostgreSQL turns a ten-minute evaluation into a project.
And the load is small. The API serves a few hundred developers doing occasional writes: signing in, creating an environment, opening a terminal. The heavy path, developer traffic to preview URLs, deliberately does not go through the API at all (ADR 0003).
Decision
SQLite in a file on a PersistentVolumeClaim, one API replica.
SQLite allows exactly one writer. Two replicas on one volume would corrupt nothing (WAL
locking prevents that) but would serve inconsistent reads and fight over the lock, so the
chart refuses api.replicas > 1 in values.schema.json rather than letting someone
discover it.
The blast radius is bounded on purpose. An API outage stops sign-in, environment creation and the dashboard. It does not stop: running environments, preview URL traffic, header-based routing, or the operator reconciling. Those were designed to run without the control plane precisely so that this decision would be affordable.
Recovery is a pod restart. The PVC carries the data; Kubernetes reschedules the pod. Realistically that is tens of seconds on a healthy cluster, and minutes if a node is gone and the volume must reattach. That is the RTO, and it should be stated to customers as such rather than implied to be zero.
Backup is a first-class command, not a file copy. nazeel-api backup --out uses
SQLite's VACUUM INTO, which is safe while the database is being written to; copying the
file underneath a running process is not. verify-backup --in opens the result and checks
the tables, because a backup nobody has restored is a hope. The chart ships a CronJob.
The PVC survives uninstall (helm.sh/resource-policy: keep). It holds an append-only
audit log. A mistyped helm uninstall must not be able to destroy an auditor's evidence.
Consequences
- Install has no external dependency: one chart, one volume. This is the single biggest reason an evaluation succeeds in an afternoon.
- The API is a single point of failure for the control plane, and we say so. Planned disruption is covered by a PodDisruptionBudget; unplanned node loss means a reschedule. Customers who need better than that need an HA story we do not have yet.
- No read replicas, so every report is computed on the one instance. The reports are small aggregate queries over thousands of rows, not millions.
- Growing past this means PostgreSQL and a migration. The store package is already behind an interface and the SQL is ordinary; the work is real but not architectural. Doing it now would be paying that cost before any customer has asked.
- A customer who insists on external HA storage today can point the PVC at their own storage class with replication underneath. That covers volume durability, not API availability.