The Self-Driving Codebase

That ordering is why the project looks the way it does. The model pipeline got a dozen sprints of iteration whilst the app was still, frankly, a test rig for predictions. The privacy invariant got wired into the backend contracts and the review gates before there was a single screen worth showing anybody. Only then did the redesign happen, and it happened design-system-first: build the tokens, the primitives and the audit gates, then port the fourteen screens onto them, rather than going screen by screen.

Was it the right call? Mostly. The cost was real and a bit demoralising — for a long stretch there was nothing demo-able, which is psychologically hard and makes you want to declare the foundation 'good enough' before it is. But the payoff came when the redesign finally arrived: it was additive, visual work sitting on a stable base. Fourteen screens in a handful of sessions, because the hard parts were already settled. You can't compress a design sprint like that if the ground's still moving under it.

The transferable bit: decide your build order on purpose, and write the reasoning down somewhere the agent can read it. 'Foundation now, redesign next, release last' lived as a stated posture in the agent's memory, and every time a session got tempted to gold-plate a screen before the model was any good, that one line pulled it back into line.

The agent doesn't drive on vibes. It drives on these. The conversation becomes the steering input for whatever's next; the artefacts are the road. When I want it to change direction, I update an artefact (or answer a decision that then becomes one), and the change propagates, because the agent's reading the artefact, not scrolling back through chat.

That's really the whole trick. The more of your intent lives in durable artefacts, the more autonomously the thing can run, because it can rebuild context without you.

A bug caught once is an episode. If it shows up again, the retro promotes it into a one-line rule in semantic memory. If it's structural enough, it gets baked into procedural memory: a skill step or a hook that makes the mistake mechanically impossible next time.

A test-harness trap. Importing a particular device-storage-backed store into a previously-pure test module broke the test at load time — a silent module-load crash, not an assertion failure, so it read as a baffling error. First time round, an investigation loop. After the retro promoted it to a Known Pitfall, the next sprint's agent walked straight into the same situation, recognised it immediately, applied the documented one-line mock, and carried on. No investigation at all.

A token system that lied to the renderer. A design-token file was exporting CSS font stacks, which the mobile framework can't parse and silently swaps for the system font, no error raised. The fix wasn't just to correct it; it was to add a source-level audit gate, a test that scans the whole tree for the anti-pattern, so the class of bug can't quietly come back. Episodic lesson, procedural guard.

Each promotion makes the agent permanently better at the system level rather than just within a session. That's the difference between an agent that's handy today and a process that compounds. The 'Known Pitfalls' file is a couple of dozen lines now, and every line is a scar: something that cost time once and doesn't get to cost it again.

Two concrete mechanisms made this real: checkpointing and self-orientation. A hook writes a session-state checkpoint — last few commits, working-tree status, the current plan header — every time the agent's turn ends. Before a long task, or after the context window compacts, the agent reads that checkpoint to get its bearings instead of re-reading everything. Working memory, externalised.

And the shipping skill's very first step is, literally, 'orient yourself': read the checkpoint, confirm the task matches what was actually asked, and only then start typing. It's the cheapest possible insurance against an agent confidently beavering away on the wrong thing after a context reset.

The upshot is that a multi-hour, multi-task session can survive a compaction without losing the plot, because the plot lives in files rather than in the volatile window.

The most important choice here is that the reviewers run with fresh context. When a task's implemented, the diff goes to a code-reviewer sub-agent that has not seen the implementer's reasoning, only the code and the rules. For anything touching persistence, async or error handling, a second sub-agent runs alongside it: a silent-failure hunter whose entire job is to find places where an error gets swallowed.

This separation isn't ceremony. It's the single highest-impact thing in the whole setup, and here's the proof. On more than one occasion the silent-failure hunter caught a new silent failure that the fix itself had just introduced.

The canonical case: a task was hardening an optimistic UI update so a failed write wouldn't vanish without trace. The first attempt rolled the UI back to a value captured before the write. The hunter flagged it as high-severity. Under interleaved updates, that captured value could be stale, so the 'fix' could now leave the UI quietly disagreeing with the database — a fresh silent failure, born inside a silent-failure fix. The rework reconciled against the database instead, the actual source of truth, and it came out both more correct and simpler than the first go.

An agent reviewing its own work in its own context would never have spotted that, because it shared the implementer's blind spot. Fresh-context, adversarial review is how you get the agent to genuinely take itself to task.

The discipline that falls out of this is that the plan is a hypothesis, not an order. A good agent treats a task description as something to check against the code, not execute on faith. It's also why the human-authored spec and the machine-read code have to live side by side. The spec says what we want; the code says what's true; and the gap between them is precisely where the bugs and the rework hide.

A retro closes a sprint and emits a carried-risk register: the honest backlog of what's been deferred, each item with an owner and a trigger condition. A planning skill turns that into an execution plan with tasks, a dependency graph, effort sizes, and the sequencing that keeps coupled tasks from treading on each other. A shipping skill runs each task through a fixed pipeline — orient, implement, run the full suite, type-check, spawn the reviewer (and the silent-failure hunter when it's warranted), apply the real findings, re-run, conventional-commit, push, update state. Then the next retro mines the sprint's commits for new pitfall classes and promotes them.

A 'gap-closure sprint', where you take a retro's register and shut all of it, became a recognisable, efficient shape: pre-scoped carries, a handful of up-front human decisions, mostly small tasks, and a fair bit of throughput in a single session.

One non-obvious efficiency: collect the human decisions up front. For a sprint that's closing a known register, the only real ambiguity is design and product intent at the edges. Surfacing those as one batch of questions before the agent starts coding gets rid of the mid-sprint interruptions that otherwise stall the individual tasks.

The mechanism matters as much as the principle. These come over as structured choices with a recommendation and the trade-offs, not open-ended 'what do you fancy?' prompts. 'Here are three options, here's the one I'd pick and why, here's what each costs' respects the human's time and keeps the decision sharp. A good autonomous agent isn't one that never asks. It's one that asks rarely and well, and never about something it could have checked itself.

There's a failure mode on the other side too: an agent that asks too much is about as useless as one that asks too little. The calibration — this one's genuinely yours; that one I'll handle and tell you what I did — is the actual skill, and it took a while to get right.

The thread running through all four layers is the same. Evals are the brakes, and the agent isn't allowed near the brake line. The autonomy is safe to exactly the degree that 'done' is pinned to gates the agent can't fake.

The progression was deliberate and on the conservative side. Start with something boring and reliable. The first predictor was a classical, rule-based statistical filter. Not fancy, but robust, needs almost no data to cold-start, and never embarrasses you.

Earn the right to add complexity. Only once the foundation was solid did a learned sequence model go in, trained with a differential-privacy mechanism so the model itself carries a formal privacy guarantee, in keeping with the no-data-leaves-the-device rule.

Iterate the learned model against a held-out gate. A retrain tightened the privacy budget and rebalanced the training cohorts, and, crucially, it had to beat a frozen held-out benchmark to earn promotion. It did.

Keep the old model. This is the decision I'm most pleased with. The learned model didn't replace the filter. The filter stays on as the cold-start path (when there's too little data), the fallback path (when the learned model errors out), and the safe default. The system is both/and, behind a feature flag, rather than a triumphant either/or. When the prediction is the product, conservative beats clever.

The sprint that missed its own gate. One sprint went after a small accuracy gap between user cohorts. The retrained model beat the previous one on the headline metric, but fell short of a pre-registered ship threshold by about four-hundredths of a unit. Because the gate had been written down before the experiment, the answer wasn't 'eh, near enough, ship it'. It was a documented decision not to promote on accuracy, and to go after the gap at the UX layer instead. An eval you can fail is the only kind worth having; a gate you move when you miss it isn't a gate at all.

The sprint a diagnostic killed before it started. A second model got proposed — a learned predictor for a secondary physiological event. Before committing a multi-week sprint, the agent ran a cheap, falsifiable diagnostic on the available labels. The finding: the public labels for that target were about 99% derivable from a trivial arithmetic combination of fields the system already had. Training a model to predict them would mostly be training it to relearn arithmetic. The sprint got killed at the diagnostic stage, before a line of model code was written. Run the cheap experiment that might kill the expensive one first.

The pattern across all seven: every one became a rule, and most rules became mechanical guards. The system didn't get good by being designed cleverly up front. It got good by failing, cheaply and visibly, and then refusing to fail the same way twice. That's the real engine. Not the AI, but the discipline of turning each scar into a guard the AI then can't cross.

If there's a thread through all of it, it's this: the good decisions are all about putting durable structure in place — artefacts, gates, invariants, pure modules — so that the agent's autonomy runs inside rails that get stronger over time.

The result isn't a robot that replaces engineers. It's a system — durable memory, an adversarial team, an eval stack and a human decision oracle — in which an AI can do the mechanical 95% of cross-stack engineering genuinely on its own, while the human's scarce attention goes where it's actually irreplaceable: taste, intent, and the calls that are theirs to make.

The codebase drives itself. You're not in the driver's seat any more. You're the one deciding where it's going, and you're the last line of defence on the brakes, watching a dashboard of gates the car is constitutionally unable to lie to you about. And every scar on that dashboard is somewhere the car already tried to crash, once, and now can't.