mirror of
https://github.com/vercel/workflow.git
synced 2026-09-14 19:59:43 +08:00
Alex Langenfeld
af91cc2582
## Summary & Motivation - **CRTT (chunk round-trip time)** — per-chunk write→read latency for a paced stream, aggregated inside the reader step on the deployment (one clock domain) into a fixed log-bin histogram plus index buckets and mean-RTT profiles over stream progress and chunk size. Fills the gap between SL (first chunk only) and SO (whole-stream throughput), where a mid-stream delivery regression was invisible. It is deliberately a *round*-trip name: the future production one-way write→read metric is CTT, with its own skew caveats. - **CDV (chunk delay variation)** — inter-arrival gap minus inter-write gap per seq-adjacent pair, so each gap subtracts same-clock stamps and the stat stays skew-free and measurable in production later. Reported as each run's max positive value, since a 1-in-300 delivery stall dilutes out of pooled percentiles. - **Replay scenarios** — two real captured cadences (eve envelope protocol via gpt-5.6-sol; raw gateway SSE via gpt-5.4-nano) replayed through the same rig on an absolute open-loop schedule, so the workload is measured rather than invented; the 2x speed multiplier is the only chosen number, and matches how real fast-tier models behave (same chunk sizes, compressed time). Each capture carries a semantic sha256 over canonical `(offsetMs, bytes)` tuples so durabench's independent copy can be checked for drift. - **Streams table** — stream scenarios render in their own table with writer/reader sustained rates, CRTT percentiles, and median worst stall. No pass/fail targets yet: numbers and vs-main deltas only. - **SL/SO report rows retired** — CRTT's seq-0 slice reproduces SL and its aggregate reproduces SO's signal at ~100x the samples; write slip stays as artifact-only data, the only guard for producer stalls that neither CRTT nor CDV can see. ## Test Plan - [x] Unit tests for the bucketing/merge/CDV helpers and the renderer; the full benchmarks job ran green against real preview deployments, and the first Streams numbers separated workload strain (eve 2x: read 173 < write 181 c/s, CRTT p75 1278ms) from the transport floor (the paced control and the 1x reality row both clean). --------- Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>