A managed LLM gateway adds a hop between your code and the provider. The honest question is: how much does that hop cost in latency and reliability? This page is our methodology for measuring it, and a commitment to publish real numbers, not a fabricated leaderboard.
What this page measures
Read this first. The latency figures below are SLA targets we hold ourselves to — not the results of a published study. A full reproducible gateway-latency report is on the roadmap. We would rather state that plainly than print numbers we cannot defend. For current component posture see the status page.
Gateway time on top of the provider call
Contractual: see the legal SLA
Switch models with a string, no re-integration
A target, not a measurement. The p50 figure is a static design target held in our source; no published latency study stands behind it yet, and we quote no tail percentiles until one does.
Routing a request does not change the model's answer. So nRouter benchmarks the things routing actually affects: added latency, failover, and throughput. Model quality stays where it belongs: with the model provider and independent evaluators.
The single number that is genuinely ours: how many milliseconds nRouter costs on top of the provider round-trip. A published study will report it at p50/p95/p99 and break it down by stage (auth, credit reserve, guardrails, routing, settle), so the number is auditable, not a black box. A pass-through request (no guardrails) is the floor case. No such study is published yet.
When a provider returns a 429, 503, or 529, the routing engine retries the next model in the fallback chain, bounded and budgeted per provider. A study reports how often that fires and how much latency the retry adds, separately from the happy-path figure. A failed attempt costs zero credits (reserve+settle).
Sustained requests-per-minute and tokens-per-minute the gateway holds before rate limits engage — the number behind the per-tier RPM/TPM guarantees on the pricing page, including the latency degradation curve and 429 behavior at the ceiling. Requests are not queued at the ceiling; they are refused with a 429 that names its source.
Model quality (MMLU, HumanEval, GPQA, reasoning scores) is the model provider’s to measure and publish. Routing a request does not change the model’s answer, so we run no model leaderboard and invent no quality scores. Honest gap: a published latency study is on the roadmap, not done.
nRouter adds no separate network service and no second hop on the path to a provider. Browse the live catalog on the models page.
A benchmark is only worth printing if someone else can reproduce it. Every study we publish follows the same four rules, and ships the config so you can run it yourself.
Run against the zero-cost mock provider so provider latency is constant. Whatever moves is gateway overhead. Nothing else.
Discard cold-start requests, then collect a large fixed sample at steady state. Cold starts are reported separately, never blended into p50.
Publish p50, p95, and p99. Never just the average. A gateway that looks fast at p50 and stalls at p99 is a slow gateway.
Every run ships its config: region, instance size, guardrails enabled, concurrency, sample size, and date. A benchmark you cannot reproduce is marketing.
A dated report, not a moving number. Each study is timestamped and versioned. Older reports stay live so you can see the trend, not just today’s headline.
Full run configuration. Region, instance size, concurrency, guardrails enabled, sample size, and provider mix — everything needed to reproduce the run.
p50, p95, and p99 — always the tail. Every latency claim ships all three percentiles. A single average is not a benchmark; it is a hope.
Failover and error rates alongside latency. Reliability and speed are reported together. A gateway that is fast only when nothing fails is not actually fast.
Want the methodology in detail, or to run a benchmark against your own workload before committing? Email sales@nrouter.ai.
The 99.9% uptime SLA is a contractual commitment in the legal SLA, not a figure we picked because it looked good on a slide. The gateway runs on managed autoscaling container infrastructure and managed Supabase Postgres.
99.9%
Committed, monitored, and public — component health and incident history are never behind a login.
Measure it yourself
Point a real workload at nRouter and watch the overhead in your own dashboard. No fabricated leaderboard to trust. Just the gateway, your requests, and the numbers.