Send one curated prompt to several LLM providers at once and compare time to first token, total time, tokens per second and cost.
◇Live mode is off on this server, so you are seeing a recorded sample run. Its numbers are illustrative, not measurements.
Prompt sent
“In one sentence, what does an API gateway do?”
Providers
○ AnthropicNot configured
○ OpenAINot configured
○ Google GeminiNot configured
An owner can switch on live mode by setting provider API keys on the server.
Recorded sample run
Illustrative numbers for small, fast models. Nothing here was measured live.
Time to first token
Anthropic
410 ms
OpenAI
380 ms
Google Gemini
450 ms
Total time
Anthropic
690 ms
OpenAI
720 ms
Google Gemini
640 ms
Anthropic
claude-haiku-5-5
First token
410 ms
Total
690 ms
Output tokens
31
Tokens per second
111
Cost at list price
< $0.0001
An API gateway sits in front of your services and handles routing, authentication, rate limiting and observability for every request, so each service does not have to.
OpenAI
gpt-4.1-mini
First token
380 ms
Total
720 ms
Output tokens
29
Tokens per second
85
Cost at list price
< $0.0001
An API gateway is a single entry point that routes client requests to backend services while enforcing security, rate limits and monitoring.
Google Gemini
gemini-2.5-flash-lite
First token
450 ms
Total
640 ms
Output tokens
27
Tokens per second
142
Cost at list price
< $0.0001
An API gateway acts as the front door for APIs, routing requests and applying policies such as authentication and rate limiting.
Recorded sample run, illustrative numbers. Anthropic: first token 410 ms, total 690 ms. OpenAI: first token 380 ms, total 720 ms. Google Gemini: first token 450 ms, total 640 ms.
Latency is the number users feel first. Pick a prompt, run it against each configured provider in parallel, and compare how long each takes to start answering, how long it takes to finish, and what the answer cost at list price. Every number is measured on this site's server, so your own connection does not skew the comparison. When live mode is off, the page shows a recorded sample run instead: its numbers are illustrative values for small, fast models, not measurements.
What the numbers mean
Time to first token is the wait before any text appears. It covers the network, queueing at the provider and the model reading the prompt. For short answers it is most of the total.
Total time runs until the last token arrives. Longer answers make generation speed matter more than the first token.
Tokens per second counts output tokens after the first one arrived, which is how fast the model writes once it has started.
Cost multiplies the reported input and output tokens by each provider's published list price. Real bills depend on discounts, caching and batch pricing.
Why one run is not a benchmark
Latency varies. Region, time of day, load and the model version all change the result. A fair comparison takes many runs and looks at percentiles, not a single sample.
Different models, different tokenisers. Token counts for the same text differ between providers, so compare cost and speed rather than raw token totals.
Small, fast models only. Each provider uses a low-cost model and a short output cap. Larger models trade speed for capability.
What a gateway adds
One place to measure. A gateway in front of every provider records the same timings for all traffic, not just a demo.
Routing on evidence. With latency and cost per provider in hand, you can route by price, speed or fallback when one provider slows down.
Limits and budgets. This page caps runs per visitor and per day. In production the same controls belong at the gateway, per team and per application.
Read more
API Gateways for AI Workloads→How API gateways are evolving to handle the unique demands of LLM traffic — streaming, token limits, and semantic caching.