No sprawl, no drift
This is the main difference. The factory checks its own quality on every train, against the same standardized metrics, and quality is only allowed to hold or improve. AI coding tools tend to let a codebase sprawl and drift; this one holds the line.
As of 18 October 2026Based on 195,000 lines of code and more than 30 billion tokens burned.
Where drift comes from
When agents write the code and another AI decides whether it is good enough, nothing holds the line. Each change is judged on its own and looks fine.
Components get written again instead of reused, patterns multiply, and quality slides a little with every merge. Nobody notices until the codebase is hard to change.
Metrics are the gate, not an AI
Opinions find problems. Numbers decide what merges.
- Gate
- A change merges when standardized metrics say so. The same metrics apply to every car of every train, so the numbers compare across time.
- Reviewers
- AI reviewers still read every change. Every finding must quote the code it is about, is checked against that code, and only then counts as real or false. Reviewers do not get the final word.
- Isolated runs
- Tests, lint and Prettier run in isolation on every change, so a result does not depend on whose machine it ran on.
Measured on every train
Six free static analysers, coverage and type checks make up a scorecard that runs on every train. A change may not add a violation.
| Metric | First measured | Now | Direction | Industry reference |
|---|---|---|---|---|
| Line coverage | 84.75% | 88.16% | ▲ better | Google: 75% commendable, 90% exemplary |
| Branch coverage | 79.44% | 82.50% | ▲ better | |
| Duplicated lines (jscpd) | 1.56% | 1.42% | ▲ better | SonarQube default: at most 3% in new code |
| ESLint findings | 258 | 253 | ▲ better | |
| Unused exports (knip) | 65 | 62 | ▲ better | |
| Vulnerable dependencies, high (OSV) | 13 | 5 | ▲ better | Critical: 0 |
| Semgrep findings (errors) | 65 (0) | 61 (0) | ▲ better | |
| Import cycles | 6 | 6 | = held | |
| Files over 800 lines | 8 | 8 | = held | |
| Files under type checking | — | 19.4% | new | Grows with every car |
A ratchet on code quality
Each quality metric has a floor that only moves up. Over many trains, quality can only accumulate.
- Holds
- The change keeps every quality metric where it is. It can merge.
- Improves
- The change makes a metric better. It can merge, and the new level becomes the floor.
- Slips
- The change adds a violation or lowers a metric. It does not merge until the metric is back.
- Lines
- Functions
- Branches
Show as table
| Day | Lines | Functions | Branches |
|---|---|---|---|
| 1 | 84.8% | 81.5% | 79.4% |
| 2 | 84.8% | 81.5% | 79.5% |
| 3 | 85.6% | 82.3% | 80.0% |
| 4 | 87.0% | 83.8% | 81.2% |
| 5 | 87.2% | 84.0% | 81.4% |
| 6 | 87.3% | 84.2% | 81.5% |
| 7 | 87.3% | 84.3% | 81.5% |
| 8 | 87.4% | 84.3% | 81.5% |
| 9 | 87.7% | 84.5% | 81.8% |
| 10 | 87.7% | 84.5% | 81.8% |
| 11 | 88.2% | 85.0% | 82.5% |
Every finding checked against the code
Each car gets two or three independent AI reviews, the whole train one more at the end. Only confirmed P1/P2 findings hold a car.
| Review | Previous week | This week |
|---|---|---|
| Review runs | 440 | 686 |
| Findings confirmed real | 337 | 298 |
| of them P0/P1 (would have hurt a user or their data) | 42 | 20 |
| Findings judged false | 41 | 52 |
| Precision (real ÷ real + false) | 89% | 85% |
| Real findings no other reviewer raised | 275 | 252 |
| Real findings per car | 2.8 | 1.9 |
Show as table
| Day | Real findings |
|---|---|
| 1 | 24 |
| 2 | 11 |
| 3 | 73 |
| 4 | 46 |
| 5 | 118 |
| 6 | 40 |
| 7 | 25 |
| 8 | 18 |
| 9 | 58 |
| 10 | 26 |
| 11 | 70 |
| 12 | 80 |
| 13 | 46 |
What a defect costs
A defect caught before merge is the cheapest defect there is. The factory knows exactly what each one cost.
- Human review, 200 lines an hour$64.00
- Human review, 400 lines an hour$32.00
- Rational Trains: review and tests$1.48
- Rational Trains: review only$1.11
Human review: the week’s 112,558 changed lines at SmartBear’s 200–400 lines an hour and €30 an hour, divided by the same 298 defects, assuming a human would find as many.
Show as table
| Item | Per defect |
|---|---|
| Human review, 200 lines an hour | $64.00 |
| Human review, 400 lines an hour | $32.00 |
| Rational Trains: review and tests | $1.48 |
| Rational Trains: review only | $1.11 |
- Second car reviewer · 70% precision$2.39
- Gate reviewer (holds the merge) · 86% precision$1.45
- Third car reviewer, on trial · 76% precision$0.47
- Train-wide reviewer · 97% precision$0.47
The train-wide reviewer brought 43% of all real findings and 13 of the 20 P0/P1, at 97% precision. A reviewer stays for the unique real P1/P2 it finds, not for its price.
Show as table
| Item | Per real finding |
|---|---|
| Second car reviewer (70% precision) | $2.39 |
| Gate reviewer (holds the merge) (86% precision) | $1.45 |
| Third car reviewer, on trial (76% precision) | $0.47 |
| Train-wide reviewer (97% precision) | $0.47 |
It names its weak spots
The same report that counts the wins lists what is not good yet, without polish.
- Flaky tests
- Green cloud runs fell from 93% to 89%. 20 test files both failed and passed on the same commit, mostly timeouts on a slow runner.
- PR size
- Median 570 changed lines against LinearB’s elite 105. On large diffs reviewers opened only 44–55% of the changed files.
- Type checks
- 19.4% of files. Every file a car touches gets checked, so the share grows with the work instead of in one migration.
- Large files
- 8 files are longer than 800 lines. Splitting them is queued, because review quality drops on them.
Everything in BigQuery
Every car and every train writes its metrics to BigQuery, so progress is visible, not claimed.
See the weekly report- Functional
- What shipped, train by train: cars, review rounds, test runs, pass rate, real and false bugs, cost.
- Code quality
- The scorecard of every train, including how much code is reused instead of written again.
- Your view
- It is plain BigQuery: query it, chart it, compare any two trains.
It learns
The same way a good team does: by looking back at every release.
- Retrospectives
- Every train ends with a retrospective. What it finds goes into how the next train runs.
- Critical bugs
- A critical bug found while a feature is being built is fixed in the same train, before release.
- Triage
- Reviewers over-report: of 278 “known issue” lines left by 13 trains, 3 were worth fixing. Triage is now part of the factory, so noise does not pile up.