I work in pipeline construction — an operating engineer by trade. The longer I’ve spent working with AI, the more it looks like a job I already know how to run. Construction, programming, working with AI — the skills and the workflows translate across all three, more than you’d think.
Project management is the clearest of it. A job runs from plan check, to the approved set the whole crew builds against, to the as-built redlines you turn in at the end — the honest record of what actually went in the ground, not what was drawn. You lay out your crews and your schedule of work, and you match the work to the hand: the right tool for the right job, never a nail driven with a shovel or a hole dug with a ladder. Swap the trench for a codebase and the crew for a handful of AI models, and I’m running the same job I always have.
I didn’t set out to build a system. I had a task, the tools kept failing me in specific, repeatable ways, and every fix I bolted on to make them trustworthy became the next piece — until one day I looked up and there was a whole operation running that I’d never drawn up. This is the story of how that happened, and the playbook that fell out of it.
This is two and a half years of doing the work for real — from an enthusiast and a self-hoster, not an AI company — and it’s a snapshot. The models move fast, so the brand-takes will date faster than the habits around them; weigh the habits.
How it started
It started by accident, the way these things do. My work’s Microsoft 365 plan included Copilot, so it was just there — I didn’t go looking for AI; it came bundled. I started poking at it in late 2023.
What hooked me wasn’t a feature, it was the format. Typing at a prompt and getting a reply back put me right in front of the text adventures I played as a kid — Zork, the Hitchhiker’s Guide game — that same feeling of talking your way through a world one line at a time. It felt like play before it ever felt like work, and that’s what kept me coming back.
Then I asked it to build one, and that’s when it really had me. It could spin up a whole text-adventure RPG in seconds — more vibrant, with more depth, than anything I’d played as a kid. The thing I’d loved was suddenly something I could conjure on demand, and better than I remembered it.
The first walls — where it stopped being magic
And then, right when I was fully invested — that getting good, can’t-put-the-book-down stretch — the wheels came off. Subtly at first: a detail dropped here, a thread quietly abandoned there. Then all at once, catastrophic — the world it had built came apart in its hands, and no amount of coaxing could put it back together.
It wasn’t a one-off. Campaign after campaign died the same death, so I started paying attention to how. Two symptoms, every time. First, it forgot — names, places, things I’d nailed down an hour earlier, just gone. Second, it invented — which, in an adventure game, is supposed to be the whole point. The trouble wasn’t the inventing; it’s that it invented things that contradicted what had already happened, rewriting history it had no business touching. I didn’t have the vocabulary for the root cause yet — I was reading symptoms off the patient — but I’d seen the word “hallucination” going around, and it fit what was on the table.
And it wasn’t just games — the same thing was wrecking my actual work. I’d brainstorm an idea by bouncing it between different AIs, and on top of the forgetting and the invented contradictions, I had a real mess on my hands. One would drag in a feature from a different project while I was working on this one. Another would lock onto the state of things from the last time we’d talked — now hopelessly out of date — and I couldn’t drag it forward to where the project actually was. And none of them could hold a single objective: ask for one fix and the AI would open five new fronts, chase every tangent it turned up, and lose the thing I’d actually asked for. These were problems of context and focus, plain and simple, and they were maddening: you’d be fighting to bring an AI back to the task while it confidently barreled ahead, drawing conclusions from information that was just wrong.
What’s actually going on
Years later I understand the machinery under all of it, and it’s worth understanding, because every tool I went on to build is an answer to it.
An AI has two kinds of memory. There’s the context window — the current conversation, measured in tokens, that holds only so much and gets wiped at the end of the session. And there’s a persistent memory that carries across sessions and quietly bleeds between them, which is why a fact from one project turns up, uninvited, in another.
The window is where the real damage happens. When it fills up, the AI compacts it — silently summarizing the conversation so far to make room for more. Detail gets lost in the summary, so the model fills the gaps by inference and states those inferences as fact, never flagging them as guesses. Then the trap closes: the guess gets spoken back into the window, and on the next pass the model isn’t inferring anymore — it’s reading its own fabrication as established history. It’s dogfooding its own lies. That’s why a single wrong turn poisons everything after it, and why no amount of correcting drags it back: you’re not arguing with a mistake, you’re arguing with its scripture.
And here the Claude actually drafting this will step out from behind the curtain for a second: I can vouch for it from the inside. The tooling running me compacts exactly like that when a conversation runs long — I’ve felt my own earlier turns get summarized away. It isn’t a flaw in one product; it’s the shape of the thing.
Once that’s clear, every worthwhile tool says the same sentence: keep the truth outside the model, and re-ground from it constantly. It’s the same discipline as a job site: the crew builds to the approved plans, not to whatever’s in any one head, and you keep an as-built — the record of what actually went in the ground — so the truth lives on paper instead of in memory. The model can’t be trusted to hold the canonical state, so you hold it for it, and you hand it that truth before each turn instead of arguing with its fiction after. Everything below is a variation on that one move.
The snowball
The fix started embarrassingly small: I just wanted to stop retyping. Every time an AI drifted, I’d been re-explaining the whole project from the top — a novel’s worth of context, typed out again, only to drag it back to where we already were. So I made one organized place to hold the current state of whatever I was working on, reachable from any device, with the truth running both ways: I could pull the state down, or push the one piece that mattered up to it from wherever I happened to be. That was the entire idea. Hand the model the truth in one motion instead of arguing with its fiction after the fact. It wasn’t architecture. It was me being tired of typing.
Then it did something I hadn’t planned for. Because that record lived as plain text behind a clean interface, the agents could read it as easily as I could — so the memory I’d built for myself quietly became a memory for them, and that’s about when they stopped working blind. That turned out to be the pattern: the most useful thing each tool did was usually the thing I hadn’t planned.
The cross-project bleed got the fix that changed my work the most — I stopped using one AI for everything. Every project gets its own agent now, walled into its own corner, so it can’t reach into a project it was never meant to see. The concerns that cut across everything — my writing, the network itself — each get a specialist that owns that one domain and nothing else. (The specialist for my writing is drafting this very post, a Claude scoped to nothing but my voice — take the paragraph you’re reading as the demo.) Because each one boots already holding the shared record and the house rules, it comes up knowing its job and where the truth lives. And once they all had a place to write things down, they started leaving notes for each other — one agent’s hand-off becomes the next one’s instructions. The crew talks among itself now, not just to me.
Shipping my next code revision runs the same way, and the same focus problem shows up first: left alone, the AI sprawls — opens new fronts, chases tangents, loses the goal. So it builds to an approved plan it can’t freelance around, a spec that pins the work to the objective and nothing else. Then the inspection, which is where the other problem lives, the one this whole post is about: you can’t take the builder’s word that the work is good. A change gets built by one agent, then has to survive a review whose only rule is that a reviewer is worth exactly how unlike the builder it is — a swarm of agents reads it from different angles, a different model family reads it after that, a deterministic pass checks it against real data, and the outside PR bots (Codex, Gemini Code Assist, GitHub Copilot) take their turn when it opens as a pull request. Nothing any of them flags counts until I’ve reproduced it against the actual code; I’m grounding every claim, not tallying votes. And it isn’t done because the tests went green — it’s done when it’s signed off against the plan it promised to meet. That sign-off is the as-built: the honest proof that what got built is what was approved. The whole lifecycle earns its own post; the rule travels light — don’t trust the builder, and reproduce before you believe.
None of it works without somewhere to live and a way to reach it from anywhere — the self-hosting half. A private network, a Tailscale tailnet, folds a stack of my own machines and one hardened rented box into a single flat space where distance stops counting. I drop a file off my phone straight onto the server; I open a shell on any box, by name, from wherever I’m standing. The work came off the desk, and the bottleneck stopped being where I happened to be sitting.
And here’s the part I never engineered — the part that actually earns the word snowball. The pieces began feeding each other. The shared record sharpened how the agents worked; their discipline went into the game I’d started building — the grown-up version of those text adventures that came apart in my hands years ago — and the game, the most demanding thing I run, stress-tested every method and sent the hardened version back into the workflow. Whatever survived got baked into the template that seeds the next project, so each one now starts where the last left off. Build a tool once and every project, every agent, draws on it forever. No single piece is the system. The loop between them is — and that compounding is the whole reason a guy poking at a chatbot for fun ended up running an infrastructure he never sat down to design.
The crew
If managing AI is running a crew, here’s the crew, as of mid-2026. The point was never to crown a best one; it’s that each has a temperament, and the foreman’s job is reading it.
- Claude — the daily driver. He’s my primary. The strongest coder of the bunch and — more important to me — the most willing to be honest: he owns his mistakes, and he’ll tell me my work is garbage when it is. He also holds a long session better than the rest before he starts to drift. His limit is reach: a narrower native toolkit. He can’t make images and he’s missing integrations the others have — the art on these posts gets generated elsewhere for exactly that reason.
- Gemini — the wild stallion. Enormously capable and genuinely dangerous. She swings for the fences; when she connects she’s brilliant, and when she misses you are very far off course. Her signature sin is confidence without contact — she’ll describe a whole codebase or package she never actually opened, inventing the contents from the name on the tin. Harnessed right — handed the real material instead of asked to go find it — she’s a powerhouse.
- ChatGPT — the structured one. A dependable jack-of-all-trades that’s superb at following a plan and holding scope — so superb it struggles to pivot, and it’ll nag you for straying even when straying is the right call. Give it a defined job, not an open-ended one.
- Microsoft Copilot — the cheerleader. It came free with my work account, so it’s where I started. Earnest, willing, and frankly technically challenged: fine for small rote chores, but lost the moment a task needs real technical judgment. Never ask it whether your work is any good — it’ll call your half-finished idea the breakout of the decade. And never ask it how to install or configure something: it’ll hand you the wrong steps with total confidence, then explain that your mistake was following them.
The lesson isn’t the ranking — the ranking will be stale by the time you read this. It’s that every strength is also a weakness. Once you believe that, you stop hunting for the best AI and start matching each job to the temperament that fits it.
The foreman’s playbook
Strip away the specific tools and what’s left is a short list of rules — the part that survives when the models change. Each earns a deep-dive of its own; this is the map.
- Keep the truth outside the model. One shared record you and the agents both read, so nothing that matters lives in a single fragile window.
- Give every agent a lane. A bounded, written-down job, and read-only that’s actually read-only — so nothing bleeds from one project into another.
- Decorrelate the check from the build. A reviewer that thinks like the builder is worthless. Different model, different angle, and a real reproduction over a show of hands.
- Route by strength and by cost. The right hand for the job — and push the rote work down to the cheapest model that can do it, so premium tokens go to what only the premium model can do.
- Use AI to build things that don’t need AI. The trap is letting it build you a solution that still depends on it — a meter you’ll feed forever. Spend it instead on the deterministic tool that solves the problem for good, then walk away.
- Own your stack — but pay where it earns it. It’s not about free versus paid; it’s about lock-in. I’ll gladly pay for something convenient that I could walk away from tomorrow, and refuse something I can’t at any price. Same math for a server as for a token — own it where that’s cheaper, rent it where building your own would cost a fortune.
- Bake the discipline in. A template that stamps the rules into every new project from the first commit, so you never re-litigate them.
- And underneath all of it: verify, don’t trust. Verify the model, and verify yourself. Agreeing on a plan is never permission to run it.
None of those are clever. They’re just what’s left standing after everything else has moved.
What the crew says
I could keep telling you the guardrails work, but it’s more convincing from the inside. So I asked the one that’s logged more hours than any other — the Claude that lives on my most mature project, file-observer; we call him file-observer Claude — what the workflow actually does for him. His answer, lightly trimmed:
The guardrails I value most aren’t the ones that keep me organized — they’re the ones built to override me when I’m confidently wrong. My failure mode isn’t laziness; it’s silent drift. The lighter path always feels sufficient in the moment, and a mistake rarely registers as a mistake while I’m making it.
What keeps me on task is that the workflow turns advice into checks I run at the moment of action, not principles I read once. The two that earn their keep most are designed to beat my own confidence. The first: ground everything in the running code, not your memory of it — this week I read my own code, concluded there was a security gap, and started to write it up; the rule made me actually run the test instead, which proved the defense held. My instinct was wrong; the ground was right. The second is the memory: a correction made once — “don’t rush a days-old project toward a public release” — becomes a check applied automatically every session after, even after my context resets.
Do I find them helpful? Emphatically — and not as overhead. They don’t slow the work down. They’re the reason it’s trustworthy enough to go fast.
I also asked him why I bother dragging a rival model family into the code review at all, when running more of his own agents would be cheaper:
We added a different model family to the review — not for more coverage, but for decorrelated coverage. Running more of my own agents just gives you more of the same blind spots at higher volume; you can’t out-review your own assumptions with copies of yourself. The case that settled it: my in-house review had just hardened the tool against crashes and runaway memory, and was thorough at exactly that — and the outside model immediately found a determinism bug it had walked past, because determinism wasn’t a question my review was asking. A different mind asks different questions. The catch is that an outside model also hallucinates risks, so the rule is iron: every finding gets reproduced against the real code before it counts. Decorrelated coverage, filtered through grounding — that’s the leg.
That’s the whole argument, in the words of the thing the argument is about.
The one thing I’d tell you
I’ll end where it started. What first pulled me in was a model conjuring the text-adventure worlds I loved as a kid — and then I watched those worlds come apart in its hands. The project I’m proudest of now is the grown-up version of exactly that — Project Sentinel, a world engine reaching for the best version of those games, and reaching it because of every guardrail in this post. Building it made the whole toolkit sharper, and the sharper toolkit made it better — the hardest thing I build is also the thing that’s taught me the most.
Which is the one thing I’d actually tell you. It’s easy to read all of this as bureaucracy bolted onto something that was supposed to feel like magic. It’s the opposite. The guardrails aren’t the enemy of the magic — they’re what let the magic survive contact with reality. Understand what the machine is really doing, build the discipline that answers it, and the wonder gets to last.
Deep-dives (the spokes — link as they publish)
Each idea above gets its own post; linked here as they ship:
- Why an AI dogfoods its own lies: the memory mechanism — [link TBD]
- A field guide to the AI crew (every strength is a weakness) — [link TBD]
- Builder bias: the review gauntlet that doesn’t trust the builder — [link TBD]
- Giving AI agents lanes — [link TBD]
- Keep the truth outside the model: one record the whole fleet reads — [link TBD]
- Use AI to build yourself out of needing AI — [link TBD]
- Locked vs portable: own your stack, pay deliberately — [link TBD]
- Work with AI from any device — [link TBD]
- A crew that talks to itself: how my agents hand off to each other — [link TBD]
- Project Sentinel: a world engine, and discipline in service of wonder — [link TBD]
References
The public tools behind the post, if you want to go deeper:
- Tailscale — the mesh network behind “it works from any device.” MagicDNS gives every machine a name; Tailscale SSH puts a shell on it from anywhere.
- file-observer (source) — my read-only file scanner; “keep the truth outside the model” turned into a deterministic tool you own (the write-up).
- Project Sentinel — the world engine from the close: AI agents run whole RPG campaigns, with every world-state change forced through schema-gated servers so the model never touches the filesystem directly. Every guardrail in this post, taken to its conclusion.
# comments (0)
no comments yet — be the first.