Fleet Series 04 — Ensembles: Many Models, One Answer

Working notes for a planned episode — an outline, not an article. Details will shift when it gets scripted. Series index: Building the Fleet.

Deep dive on the ensemble harness: fan a question out to many models, combine the answers, label the confidence. One resilient pattern, many consumers.

The hook

A single model reviewing code is an opinion. Five models reviewing independently, with findings aggregated by quorum, is starting to look like evidence. The ensemble harness is one shared fan-out+combine primitive that turned into the fleet's answer for review, testing, refactoring, and audits.

What the viewer walks away with

• The core shape: fan out the same prompt/work to N executors (different models, providers, temperatures), survive individual failures, combine into one artifact
• Quorum labeling: a finding flagged by 4/5 reviewers reads very differently than one flagged by 1/5 — and the output says which
• Thin specializations: the PR review ensemble, testing ensemble, and refactoring ensemble are all small wrappers over the same audit recipe and shared harness
• parallel-edit: same pattern for generation — N models each implement the change in their own workspace, human compares diffs and picks (generate-and-compare beats review-the-one-attempt)
• Ensembles as pipeline gates: wave verification consumes review-ensemble findings; the testing auto-merge loop runs behind 3 fail-closed gates

Beats

• Open with a real multi-model review advisory: where the models agree, where one dissents, why both are useful
• The resilience story: providers time out, models return garbage — the harness treats that as normal weather
• Generate-and-compare philosophy: why comparing N diffs beats carefully reviewing 1
• Society-of-Mind aside: an ensemble is an "agency" — simple agents composing into something none of them is (30 seconds, pointer to the blue-sky material)

Demo ideas

• Run the review ensemble on a genuinely gnarly pending diff, walk the quorum-labeled advisory
• parallel-edit with 3 models on one prompt, side-by-side diff comparison report

Source material

• The ensemble harness in the public forge repo
• The review / testing / refactoring ensemble builds
• The testing auto-merge loop — 3 fail-closed gates, shared auto-merge gate module
• Society of Mind — a Lens on the Fleet — the agency framing

Open questions

• Show real cost numbers per ensemble run? Probably yes — the economics are part of the honesty of the series (and the Cost Series will go deeper)