T3 Code · Performance over time

How it is measured.

Every number on this site comes from one suite, run the same way on every upstream nightly and on every fork release, and published with the files that reproduce it.

The suite.

Six load levels: 1, 50, 100, 150, 200 and 250 agents. At each level N the server carries N agents, 3 people watching live, switching threads every 300 s, and status polling that grows with N (N/4 pollers, rounded up, each calling 0.01 times a second); each agent waits 60 s between turns, and 5 s between turns at 1 agent. An agent is one streaming session on the server, the harness's --sessions count. The 1-agent level is one developer starting a new thread on the same large database, with 1 person watching.

At the higher levels the server may saturate: not every agent is served within the window. A level counts as saturated only when the server's own event loop was at least 95% busy; the signs are agents not served, calls timed out, and MCP not ready by the start of the window. Such an attempt on a clean host is still published, and flagged "saturated: n of N agents served" on the charts, on Records and in compare. Its latency figures are context only, never a verdict; turns completed and agents served keep their verdicts there.

The server runs on real wall-clock time; the fixture's own dates are fixed by its seed and recipe, and background maintenance that depends on dates is seeded so it behaves the same in every run.

CellAgentsWatching liveStatus pollersDatabase
1 agent110the era fixture
50 agents50313the era fixture
100 agents100325the era fixture
150 agents150338the era fixture
200 agents200350the era fixture
250 agents250363the era fixture
Fresh install110empty

From the latest record's workload, real3.

Each level runs a 60 s warm-up and a 300 s measured window, three times, the levels interleaved so that a slow minute on the machine cannot land on one level only. A seventh cell is a fresh install, one agent: an empty database, for start-up, the baseline and single-turn latency. 21 attempts per record. Start-up, migrations and the full-snapshot read are measured once per record, outside the ladder.

The machine.

Linux x64: a cloud VM (n2d-standard-16, AMD EPYC 7B13, spot), a new one for every record, running Debian 13. A record whose CPU model is not the pinned one is discarded. macOS records are not yet published; they start on a dedicated Mac mini after launch.

The database.

A synthetic fixture, written through the server's own code from a published shape and seed; nothing in it comes from any real deployment. Its shape, as fixture/public-large.shape.json states it: 50 projects, 3,000 threads, 60,000 runs, 1,800,000 turn items, 540,000 messages and 4,700,000 events. One fixture serves each schema era, and every nightly migrates its era's fixture forward once per record (the migration time is itself a metric):

The recipe, for anyone who wants to build it instead of downloading it:

node tools/server-bench/fixture.ts synth --like fixture/public-large.shape.json --scale 1 --seed 20261017 --now 2026-10-17T00:00:00Z --workspace-root /t3-bench-synthetic large-76-v1.sqlite && node tools/server-bench/fixture.ts scan --keep-content large-76-v1.sqlite

Validity.

An attempt counts only when no client call failed, none was left unfinished, the host stayed quiet (busy, steal and I/O wait under their limits) and the CPU model, harness and fixture match the series. A level needs two valid attempts for a verdict; a value from fewer is drawn hollow and gets none.

From 20 Oct 2026, an attempt during which the server's background maintenance ran inside the measured window is invalid. Builds whose maintenance pass emits no named span cannot be checked for maintenance inside the window.

Aggregation.

A level's value is the median of its valid attempts; the minimum and the maximum are kept and drawn as the range of attempts. Every attempt's value is in the record file.

The regression rule.

Each record is compared with the median of up to seven previous complete records on the same platform and level; while fewer than three exist, with the previous record alone, marked as a thin reference. The change counts only beyond the metric's noise band plus five percentage points: regressed beyond noise, improved beyond noise, otherwise within noise. The reference never crosses a fixture era or a harness version, and the first record after an era boundary gets no verdict. Metrics shown for context, and metrics without a measured band, are drawn and never judged.

On the charts a record beyond the band carries a triangle pointing the way the value moved, green when that is better and red when it is worse. A fork release is compared with its upstream base: the upstream code at the commit the fork release is based on, built from source the same way, and with the previous fork release, by the same rule.

Noise bands.

On Linux x64 the bands come from 42 identical attempts on 3 machines of the record's type, measured 15 Nov 2026 to 15 Nov 2026 on v0.1.10-nightly.20261115.4749. 78 of the 120 metrics have a band. The band is (max - min) / median in percent, rounded to 0.1, over the values of the valid attempts of all machines pooled per metric and cell: three spot VMs, two attempts per cell each, on one nightly. A metric band is its largest cell band; a cell needs at least two values and a non-zero median.

Eras.

The nightlies from 17 Oct 2026 to 19 Oct 2026 form their own era: their database compaction schedule differs from later builds'. The charts mark the boundary, and no verdict, noise band or reference median crosses it; the bands are measured in the later era.

Event-loop delay comes from two sources that are never read as one distribution. The eld_* figures are the harness's preload histogram (10 ms timer intervals, the 10 ms period included); server_eld_max_ms is the server's own gauge (a 1 s timer, the period excluded).

Each record names the preload it ran with by preload_sha256, the harness's own identity of it: the sha256 over the lines <file name>\0<sha256 of the file>\n for tools/server-bench/preload/bench-preload.cjs and then bench-preload.mjs, each when present. A run that loaded no preload records none.

Some metrics start later than the window, because the server began to emit them later. Before a metric's era its series is a gap, never a zero.

Harness versions:

Fork builds.

The fork's releases are built from their public tags on the same machine type and run through the same suite. 5 metrics exist only in fork builds; they are plotted fork release against fork release and never judged against upstream.

Every metric.

The catalogue holds 120 metrics, each with its unit, direction, band, era, lane and the harness field it comes from: Metrics. 27 of them need /proc and so are Linux only.