Skip to content

Lab

One request, eight plugins.

Follow a single AI request through an ordered chain of gateway plugins, switch any of them off and see what changes.

Educational simulation of an AI gateway. Curated requests only; latency, tokens and prices are illustrative, not benchmarks.

Choose a request

The request

Caller
Support app with a valid API key
This minute
Request 12 (limit 60)

Prompt

Summarise our returns policy in three bullet points for a customer.

Send plays the whole chain. Step moves one plugin per click, which is handy when presenting.

Gateway plugin chain, in order

Switch a plugin off, then send the same request again to see what changes.

  1. Authentication

    Checks the API key and identifies the consumer.

    Waiting

  2. Rate limiting

    Caps requests per consumer per minute.

    Waiting

  3. PII redaction

    Replaces personal data with placeholders.

    Waiting

  4. Prompt guard

    Blocks attempts to override instructions.

    Waiting

  5. Semantic cache

    Answers repeated questions without a model call.

    Waiting

  6. Model router

    Sends simple tasks to a smaller, cheaper model.

    Waiting

  7. Provider call

    Calls the model, falling back to a second provider on 429 or 5xx.

    Waiting

  8. Logging and metrics

    Records consumer, model, tokens, latency and cost.

    Waiting

An AI gateway does its work as a chain of plugins that every request passes through in a fixed order. Pick a request, send it, and watch each plugin pass it on, change it or stop it. Then switch a plugin off and send the same request again.

Why the order matters

  • Cheap checks first. Authentication and rate limiting take a millisecond or two, so unknown callers and runaway scripts are turned away before anything expensive runs.
  • Clean the prompt before you look it up. Redaction runs before the cache, so personal data never becomes part of a cache key, and the prompt guard runs before the cache so a blocked prompt can never be answered from it.
  • Route only what will really be sent. The router runs after the cache, so a cache hit never pays for a model decision it does not need.
  • Log everything, including rejections. The logging plugin runs for every request. Blocked requests are often the most useful records: they show attacks, broken clients and limits that are set too tight.

What a real deployment needs

  • Identity that flows through the chain. Rate limits, budgets, routing rules and cost reports all depend on knowing which consumer sent the request.
  • Detection you have tested. Pattern matching catches obvious email addresses and card numbers. Names in free text, unusual formats and indirect prompt injection need stronger detection and regular testing against your own traffic.
  • Fallbacks that are actually compatible. A secondary provider needs the same request format, enough context window and capacity, and has to meet any rules about where data is processed.
  • Limits on retries as well as requests. Fallback hides a provider's 429 from users, but without a rate limit in front of it a runaway client simply moves on to the next provider's quota.
  • Configuration as code. In products such as Kong, each step here is a plugin configured per route or per consumer, so the chain can be reviewed, versioned and tested like any other code.

About this simulation

It runs entirely in your browser with a fixed set of example requests. It does not call a gateway or any AI provider. Latencies, token counts and prices are illustrative values chosen to make each decision visible; they are not benchmarks or published prices.

All Lab experiences