// quality

Gates, and what they miss.

Every change to this repository clears the same path: a per-component test lane, a four-dimension score against published rubrics, and a set of named triggers that force a human to look. Here is what that path actually checks — and, at the bottom, where it is weaker than it sounds.

// counted, not estimated

Four figures from the repository.

Counted from the tree with the commands recorded in this page’s source, so anyone can reproduce them.

python test files

frontend test files

ci workflows

test lanes

counted 5 August 2026 · they grow · file counts, not coverage

// the scoring gate

Four dimensions, four thresholds.

Four rubrics live in docs/scoring/, ten criteria each, one point per criterion, no partial credit. A change is scored on all four and every one has to clear its own bar.

≥ 8 / 10
Security
No secrets in source. Pydantic validation at every HTTP, MCP and CLI boundary. Protected routes scope-checked and fail-closed when auth is unconfigured. No new loopback exception, no debug back door, and no live-trading path touched without a human gate.
≥ 8 / 10
Quality
Pydantic v2 and Polars only — no pandas. Ruff clean at line length 100. A test for every new public function or route. No file over 400 lines, no orphaned exports, structured errors rather than bare raises, and the component's ARCHITECTURE.md updated in the same change.
≥ 7 / 10
Optimization
LLM calls routed through the cached LiteLLM path with no hardcoded model strings. Polars lazy frames with a single collect. No N+1 request or embedding loops, no blocking call in an async route, and the ten-million-row backtest budget held.
≥ 9 / 10
Accuracy
The strictest threshold, because this is the dimension about being wrong rather than being untidy: correct LangGraph state transitions, an audit event for every persistent state change, no silenced error paths, unchanged public API contracts, preserved Nautilus event lifecycle, and assertions that check values rather than merely not throwing.

// how the gate runs

One half is machine-checked. Say which.

The scanner

blocking

scripts/score.py runs as a CI job against the pull request’s diff and exits non-zero when any dimension is under threshold, so the check goes red and the merge waits. Its own header is blunter than that: “a heuristic scanner — it flags known anti-patterns by regex… It is NOT a full static analyzer. Treat results as a checklist aide, not a gate.” Both halves are true and worth stating together: the script disclaims being a gate because a regex cannot judge a novel anti-pattern, and the workflow uses it as one anyway because a known anti-pattern should not need a reviewer to catch it.

The rubrics

self-scored

The forty criteria are worked through by the author in the pull-request template: check what you pass, write a note for what you do not and why it is acceptable, leave anything you are unsure about unchecked. Recorded judgement, visible in the pull request, reviewable by whoever comes next.

Thresholds are Security 8, Quality 8, Optimization 7, Accuracy 9 — the figures the scanner’s own table, the rubric index and the repository’s contributor rules all carry. The individual rubric files disagree: each one’s header adds a second, lower number below which merge is blocked. That contradiction is in the repository, not resolved by this page, and the stricter reading is the one quoted here.

// the lanes

What runs before a merge.

Sixty-three workflow files, most of them fired by path filters so a change pays only for the surface it touched.

Per-component test lanesSeventeen test workflows, most of them one per component, fired by a path filter so a change to digikey does not wait on the quant suite. Four of the seventeen are cross-cutting instead: end-to-end, the scoring job, the isolated Nautilus run, and the Atlas graph spec. Adding a component means wiring its lane; the filter is explicit rather than inferred.
An isolated Nautilus laneTests that import NautilusTrader run in their own workflow and are ignored during ordinary collection. The Rust engine initialises its logger once per process, so real-engine tests cannot share a run with everything else — the isolation is a correctness requirement, not a preference.
Type checkingA dedicated mypy workflow over the shared Python libraries — digibase and digikey — on every pull request that touches them. The frontend apps have no type-check lane of their own: they are type-checked by the production build, which CI runs as a deploy check on any pull request touching an app or the shared design packages, and which fails on a type error. Strict typing is a rubric criterion too, so an untyped signature also costs a score point.
The frontend canon guardA script that scans every tracked frontend file for raw Tailwind palette utilities, pre-canon class vocabulary and colour literals in component code, and ratchets on new app-local CSS class families. Pages have to assemble from the shared design system rather than growing private dress — this page was built under that constraint.
Documentation link checkingInternal markdown links are validated in CI, and the architecture documents are synced. A dead cross-reference in an ARCHITECTURE.md fails the same way a dead import would.
Workflow and compose lintingactionlint over the workflow files and a compose validation job, so the CI definition and the deployment topology are themselves checked rather than trusted.
Pull-request hygieneEvery change has to trace to a GitHub issue — a task branch carrying the issue number, or a closing keyword in the pull request. Documentation and chore branches are deliberately exempt, so the rule stays enforceable instead of routinely waived.

// limits

Where this is weaker than it sounds.

A quality page that only lists gates is a marketing page. These are the four things worth knowing before you take the numbers above as a guarantee.

The scanner is a heuristicThe scoring script is regular expressions over a diff, standard library only. It catches known anti-patterns — a pandas import, a bare exec, a blocking sleep in an async handler — and it will miss a novel one. Its own docstring says so, and this page is not going to say otherwise.
Half the gate is self-assessedThe forty rubric criteria are evaluated by the change's author in the pull-request template. That is a design choice with a real failure mode: an author who scores generously produces a green gate. Named human-review triggers — auth and crypto changes, broker or live-trading paths, a score below threshold twice, a new external dependency, novel architecture — exist because the self-score alone is not sufficient.
Frontend is scored differentlyThe score job excludes frontend/** entirely: the rubrics are Python-oriented and misfire on JS and CSS. Presentation work is gated instead by secret scanning, the canon guard, lint, and a production build that fails on a type error. Only two of the front ends — the chat UI and the Olympus dashboard — run their test suites in CI; the marketing sites and the shared component package have no CI test lane, so their tests are a local discipline. A narrower net, honestly narrower.
Test count is not coverageThe figures above count files, not lines exercised, and a file count says nothing about assertion quality. The rubrics push at that directly — meaningful assertions, no test deleted or weakened to go green — but a published coverage percentage is not something this page claims.

The rubrics themselves are in the repository, so you can judge the bar rather than take our word for where it sits. The security page does the same for the runtime posture.

Read the rubrics API reference