Lightning Talk — The Loop Is the Easy Part

Talk scaffolding, not a script — the working material for a ~5-minute lightning talk about the fleet's coding pipeline. The whole talk hangs on one idea; everything else is support for it. The Fleet Series expands this into full episodes.

Title options

• "The loop is the easy part."
• "What's your evaluator, and where's the gate?"
• "The day the pipeline fixed its own bug."

Abstract (the elevator pitch)

Everyone's talking about "agentic coding loops." Here's the uncomfortable bit: the loop itself is about five lines — generate a change, run the tests, repeat. That part is trivial and it's not where the value is. The value is in everything around the loop: what decides a change is actually good, where you stop and hand control to a human, and how you keep a model honest when "the tests pass" doesn't mean "the code is right." This talk walks through a working iterative coding pipeline — an architect that turns a goal into a bounded tree of small tasks, and an orchestrator that builds it in waves behind fail-closed gates — and the moment it started finding and fixing its own bugs through its own gates.

The one idea (if they remember nothing else)

The loop is trivial. The evaluator is the product. Any "agentic coding" demo is worth exactly what its evaluator is worth. So ask two questions of anything you're shown:

1. What's your evaluator? (How does it know a change is good — not just green?)
2. Where's the gate? (What is it not allowed to do without a human?)

Everything else in this talk is just those two questions applied over and over.

Talk arc (rough beats)

• Hook (30s) — "Agentic coding loop" is on every slide deck this quarter. Show the five-line loop. Say: this is the boring part, and I'm going to spend the rest of the talk on the boring part's expensive cousin — the stuff that makes it safe to run unattended.
• The frame (60s) — It's a generator–evaluator loop. The generator (a model writing code) is cheap and getting cheaper. The evaluator (does this actually work / is it safe to keep) is where all the engineering goes. Green ≠ correct — a test that passes for the wrong reason is worse than no test. [maybe: the "self-consistent inversion" failure — model writes the bug and a test that asserts the bug.]
• The two roles (90s) — Architect plans, Orchestrator builds.
• Architect: goal → inventory the repo → propose a framing (and push back if the goal's wrong) → stop for a human → decompose into small, bounded tasks → file them.
• Orchestrator: build the tree in waves — pick what's ready, do each task in a sandbox, verify, then replan based on what happened. Repeat.
• The gates (60s) — Walk one change through: tests must be green (hard), a review panel signs off (quorum), and a human always merges to main. Any gate misses → it reverts and files the work for a person. Worst case is "no progress," never "bad merge." This is the product.
• The money moment (60s) — It dogfooded its own construction. The loop authored real fixes to itself — through its own gates, zero bad commits. Best one: replan's "smart" step was silently failing every time; we added instrumentation, caught what the model actually emitted, and it was a one-word schema mismatch. Fixed it, confirmed live. The pipeline's last bug was found and fixed by the pipeline. [demo: show the journal entry / the before-and-after]
• Close (20s) — Restate the one idea. The models will keep getting better at the generator half for free. The evaluator half is the part you own. Build that.

The moving pieces (label each on a slide)

• Task tree — work is an epic → features → leaves. A leaf is one small, self-contained task: single concern, a handful of files, testable acceptance criteria. If you can't name the files, it isn't a leaf yet.
• Architect — turns a goal into that tree. Five stages: inventory the repo, frame the problem with a mandatory human sign-off, decompose into worker-shaped leaves, file them, and replan after each wave when reality disagrees with the plan.
• Orchestrator — the wave loop. Plan the ready-set → dispatch each leaf to the worker → verify → replan → journal → repeat until done or blocked.
• Worker — does one leaf in an isolated sandbox, gated by the test suite and the linter.
• Gates — fail-closed checks between "a model did a thing" and "we keep it": green suite, review-panel sign-off, human merge. A miss reverts and escalates rather than proceeds.
• Replan — the judgment step. After a wave, decide: retry with a sharper spec, spin off a fix for a real finding, or hand it to a human. This is where "green ≠ correct" gets enforced.
• Journal — every decision is written down, so a run can resume, and so a silent failure becomes a readable line instead of a shrug. (The self-diagnosis in the money moment came from here.)
• Human-in-the-loop — two non-negotiable stops: approving the framing before any work is filed, and merging to main. The machine never crosses either on its own.

Glossary

• Agentic loop — a model that acts, observes the result, and acts again. The generate→check→repeat cycle. Structurally trivial; the interesting engineering is around it.
• Generator–evaluator loop — the useful framing: a cheap generator proposes, an expensive evaluator decides whether to keep it. Quality is bounded by the evaluator, not the generator.
• Evaluator — whatever decides a change is actually good. Tests, review panels, sign-off gates. The thing that turns "it ran" into "it's right."
• Architect — the planning role: goal → framed, bounded, filed task tree. Plans; doesn't code.
• Orchestrator — the execution role: builds the tree in waves and reacts to results. Codes (via the worker); doesn't plan from scratch.
• Worker — the unit that implements a single leaf in a sandbox, behind test + lint gates.
• Task tree / epic / feature / leaf — the work, decomposed. Epic = the whole goal; feature = a grouping; leaf = one small, testable, buildable task.
• Worker-shaped — a leaf small enough to hand to an automated worker: single concern, bounded diff, testable acceptance, named files. Anything fuzzier is tagged for a human.
• Wave — one pass of the orchestrator: build everything currently unblocked, verify, replan. Work proceeds in waves because finishing some leaves unblocks others.
• Fail-closed gate — a check that, on any doubt, stops and reverts instead of proceeding. The default is "do nothing," never "do the risky thing."
• Sign-off / quorum — a panel of independent reviewers; the change only passes if enough approve. Diversity of reviewers catches what a single reviewer misses.
• Replan — the post-wave judgment call: respec-and-retry, spin off a fix, or escalate.
• Escalation — handing a task to a human because it's stuck, ambiguous, or on a risky path. A valid, expected outcome — not a failure.
• Journal — the durable, append-only record of decisions; enables resume and post-hoc debugging.
• Idempotent emission — filing the task tree so re-running only creates genuinely new tasks; makes replans cheap and safe to repeat.
• Autonomy tag — per-task setting for how much the machine may do unattended (manual vs. auto-OK), plus which model tier runs it. Routine + well-tested → auto; design latitude or risky → human.
• Green ≠ correct — the core caution: a passing suite can hide a test that asserts the bug. The evaluator has to be better than "it's green."
• Dogfooding / self-hardening — the pipeline improving itself by running its own process on its own codebase. How its last several bugs got fixed.

Slide outline (10 slides, ~5 min)

1 — Title (10s)

• On screen: "The loop is the easy part." + your name. (Faded 5-line loop behind the title.)
• You say: "Quick one about agentic coding — specifically, why the part everyone demos is the part that matters least."

2 — The hype (30s)

• On screen: the five-line loop: generate → run tests → repeat.
• You say: "This is 'the agentic coding loop.' It's on every deck this quarter. It's also about five lines, and it's the boring part. I want to talk about the boring part's expensive cousin."

3 — The reframe (40s)

• On screen: two boxes — generator (cheap ↓) → evaluator (expensive).
• You say: "It's a generator–evaluator loop. The generator — a model writing code — is cheap and getting cheaper for free. The evaluator — is this actually good? — is where all the engineering lives. Your output is only as good as your evaluator."

4 — The trap (30s)

• On screen: Green ≠ correct. A tiny passing test that asserts the bug.
• You say: "And the evaluator has to be better than 'the tests pass.' A model will happily write the bug and a test that asserts the bug. Green is not correct. That's the whole problem in one line." (Cuttable if long — but it's a strong beat.)

5 — Two roles (30s)

• On screen: Architect → Orchestrator (one plans, one builds).
• You say: "So here's the shape we built. Two roles: an architect that plans, an orchestrator that builds. Same split you'd give two people."

6 — Architect (40s)

• On screen: goal → bounded task tree, with a 🛑 human stop before anything's filed.
• You say: "The architect turns a goal into a tree of small, testable tasks — and it pushes back if the goal's wrong. But it files nothing until a human approves the framing. That stop is not optional."

7 — Orchestrator: waves (40s)

• On screen: a cycle — build → verify → replan — labelled "wave."
• You say: "The orchestrator builds it in waves: do everything that's unblocked, verify it, then replan based on what actually happened — retry smarter, spin off a fix, or escalate. Finishing some tasks unblocks the next wave."

8 — The gates (40s)

• On screen: green suite → sign-off → human merge and, below, miss → revert + escalate.
• You say: "Between 'a model did a thing' and 'we keep it' are fail-closed gates: green suite, a review panel signs off, a human merges to main — always. Miss any gate and it reverts and files the work for a person. Worst case is no progress, never bad merge. This — not the loop — is the product."

9 — The money moment (60s) ← protect this one

• On screen: a real journal line / before → after of the self-fixed bug.
• You say: "Here's the part I like. It dogfooded its own construction — the loop wrote real fixes to itself, through its own gates, zero bad commits. The best one: its 'smart' replan step was silently failing every single time. We added instrumentation to capture what the model actually emitted — and it was a one-word schema mismatch. Fixed it, confirmed it live. The pipeline's last bug was found and fixed by the pipeline."

10 — Takeaway (20s)

• On screen: "What's your evaluator? Where's the gate?"
• You say: "So next time someone shows you an agentic coding demo, two questions: what's your evaluator, and where's the gate? The generator half gets better for free. The evaluator half is yours to build. Thanks."

Backup slides (if asked): the full architect stages; the task-tree/leaf anatomy; the sandbox + worker; how sign-off quorum works; the auto-tier failure signature.

Timing check: 10 + 30 + 40 + 30 + 30 + 40 + 40 + 40 + 60 + 20 ≈ 5:40. Trim slide 4 or fold 7 into 5 to land under 5:00.