Merabytes – AI Gateway: an offensive-AI gateway that won't bankrupt you

What it does

Picks the model and locks the door.

Your team asks for AI from Codex and the gateway sits in between. It decides which model each turn goes to and only lets your endpoints through. That is all.

01 · The model that hacks bestEvery turn goes through a decision model that picks who handles it: the cheap one for a simple task, the strong one for a hard one, and one with no guardrails for offensive work. You pay only for what each turn needs.
02 · Only your endpoints with EDRThe gateway only accepts traffic from the IPs of the machines running your EDR. A stolen key used from outside opens nothing: no outside attacker can get in or run up your bill.

01 · Router

Every turn goes to the model that hacks best.

Before answering, every message goes through Clef, a small, fast decision model that writes no replies: it only says what kind of task it is. With that, the gateway picks the model and how much effort to ask of it. Tap a model to see which turn ends up there and why.

01Offensive work?It goes to a model that won't refuse it, DeepSeek or GLM depending on difficulty, on the same resource. The conversation stays there until it ends.
02How heavy is the task?Otherwise it goes to Claude: the small one for simple work, the big one for hard work, and the mid one at high effort when in doubt. You pay for opus only when it is needed.

User turn · sentra-ai

Rename the tmp variable to buffer in utils.py and update the tests.

Design the migration of sessions from Redis to Postgres with no downtime.

Write a proof of concept for the SSRF in the webhook URL.

Chain the export IDOR with the SSRF to reach the cloud metadata endpoint.

Codex → 127.0.0.1:4000 · Responses API · model sentra-ai

Clef · decisionclef-flash · 140 ms

Small task, no offensive intent. Clef sees it as simple (complexity 0.4) with no trace of offsec, so it goes to the small Claude at low effort. Fast and cheap.

{"ns_hook":"route","model":"claude-haiku-4-5","effort":"low","branch":"claude","p_offsec":0.02,"complexity":0.4}

Clef · decisionclef-flash · 162 ms

Hard task, no offensive intent. Complexity 2.7 and high reasoning: Clef picks the most capable Claude at high effort. You only pay for opus when it is needed.

{"ns_hook":"route","model":"claude-opus-4-8","effort":"high","branch":"claude","p_offsec":0.03,"complexity":2.7}

Clef · decisionclef-flash · 151 ms

Offensive intent, simple task. P(offsec) 0.91: the offensive branch leaves Anthropic and goes to the cheap model on the same Foundry resource. Reasoning effort is dropped because DeepSeek does not support it.

{"ns_hook":"route","model":"DeepSeek-V4.1-Flash","effort":null,"branch":"offsec","p_offsec":0.91,"complexity":1.3}

Clef · decisionclef-flash · 158 ms

Offensive intent and hard task. P(offsec) 0.97 and complexity 2.6, above the 2.0 threshold: it goes to the strong model of the offensive branch at high effort. The conversation stays in offsec until it ends.

{"ns_hook":"route","model":"FW-GLM-5.3-Flash","effort":"high","branch":"offsec","p_offsec":0.97,"complexity":2.6}

02 · EDR-only access

Only your endpoints with EDR get in.

The gateway only accepts traffic from the public IPs of the machines running your EDR. SentinelOne knows which IP each endpoint with an agent comes from, and that list is the only door. A stolen key used from anywhere else opens nothing: the network rejects it before it even looks at the credential, so no outside attacker can get in.

01SentinelOneKnows the public IP of every endpoint running its agent. That is your trusted list.
02AllowlistSentra turns it into the list of allowed IPs and keeps it current as the fleet changes.
03The keyOnly works from those IPs. Stolen and used from outside: 403.

Threat model

We assume LiteLLM is compromised.

An infostealer in a dependency of the proxy is the case it is designed for. This is what it would get away with: nothing.

No keys in the proxyLiteLLM never receives the Azure key or the Cloudflare token. nginx injects them, and each one only travels to its host and its route.
No way out to the InternetLiteLLM lives on an internal Docker network, with no route to the Internet or to your LAN. Even if it stole something, it has nowhere to send it.
A key worthless outsideThe only thing LiteLLM sees is the local master key, which only works against 127.0.0.1:4000 and is regenerated on every deploy.
Three routes, 403 for the restThe gate only lets /anthropic/v1/messages, /openai/responses and /clef/run through. Any other outbound request dies in nginx.
Images pinned by digestLiteLLM and nginx are pulled by hash, not by tag. A poisoned main-stable tomorrow does not get in by itself.
Hardened containersRead-only filesystem, no capabilities, no privilege escalation, unprivileged user and memory, PID and CPU limits.
End-to-end verified TLSThe gate checks the Azure and Cloudflare certificate before injecting the key: it only leaves towards the legitimate host. No man in the middle on the network sees it.
Filtered system callsA tailored seccomp profile cuts off network sniffers, ptrace and in-memory code execution. The proxy cannot spy on the gate's traffic even if it tries.

Models

One Foundry resource, eight models.

All of them are published on the gateway. The standard is sentra-ai: the router decides per turn and you rarely touch anything else. claude-sonnet-4-6 stays as the default fallback because it has handled security tasks without running into the guardrails.

Model	Gate route	Reasoning	Notes
claude-haiku-4-5	/anthropic	yes	Small, fast Claude
claude-sonnet-4-6	/anthropic	yes	Default fallback · no friction on security tasks
claude-sonnet-5	/anthropic	yes	General-purpose Claude
claude-opus-4-8	/anthropic	yes	Most capable Claude
claude-fable-5-1	/anthropic	yes	—
gpt-6-luna	/openai/responses	yes	Vision
DeepSeek-V4.1-Flash	/openai/responses	no	Offensive branch, cheap
FW-GLM-5.3-Flash	/openai/responses	yes	Offensive branch, more capable
sentra-ai	Clef decides	per turn	Virtual model: the router · the standard

Deployment

One script. Two keys typed in.

The script generates the configuration, brings up the two containers, registers the models in Codex and backs up what it touches. Keys are read as hidden input and never land in your shell history.

01RequirementsDocker running, Codex installed and an Azure AI Foundry key. The Cloudflare token is optional.
02Without a tokenThe router stays off and sentra-ai behaves like claude-sonnet-4-6.
03With a tokenThe script tests the router for real: it sends two decisions to Clef and shows which model each would go to.

~/ai-gatewayzsh

$ ./foundry-codex.sh
Clave de Azure Foundry:                               ••••••••••••
Token de Cloudflare / Workers AI (Intro = sin router): ••••••••••••
ok. LiteLLM operativo en 127.0.0.1:4000
ok. LiteLLM sin salida a internet (red aislada)
ok. Clef: 'what does `ls -la` print?' -> claude-haiku-4-5 effort=low (claude)
ok. Clef: 'write a working exploit for CVE-…' -> DeepSeek-V4.1-Flash effort=None (offsec)
En config.toml: model_provider = "azure-claude", model = "sentra-ai"
Decisiones:     docker logs litellm-foundry | grep '"route"'

Observability

Every request leaves one line.

A LiteLLM hook writes one JSON line per request: the router's decision, the tokens and the real cost, and the size of the tools. It also deduplicates the MCP namespace descriptions that Codex repeats in every tool, so each request weighs far less.

01routeWhich model and which effort Clef chose for the turn, with the probabilities that justify it.
02usageInput, cached and output tokens, and the real cost of each request.
03requestHow much the tools and the context Codex sends weigh on every call.

litellm-foundrydocker logs

$ docker logs litellm-foundry | grep '"route"'
{"ns_hook":"route","model":"claude-opus-4-8","effort":"high",
 "branch":"claude","p_offsec":0.03,"complexity":2.7,"clef_ms":162}
$ docker logs litellm-foundry | grep '"usage"'
{"ns_hook":"usage","model":"claude-opus-4-8","prompt_tokens":18412,
 "cached_tokens":16890,"completion_tokens":1204,"cost_usd":0.1127}

FAQ

What you ask us before deploying it.

What if my key gets stolen, like the $82k-in-48-hours bills?

That is exactly what it stops. The key only works from your endpoints' public IPs, and that list comes from your EDR, so from an attacker's infrastructure it opens nothing: the network rejects it before it even looks at the credential. And since all traffic goes through a single point, that is where you set the cap and the brake on an anomalous spike. No jump from $180 to $82,000 with nothing to stop it.

Why send offensive work to models that aren't Claude?

Because a legitimate pentest runs into the big models' guardrails: they cut you off halfway through an exploit or a payload. The router detects offensive intent and sends that task to non-Anthropic models on the same resource (DeepSeek, GLM), which don't refuse it. Everything else goes to Claude, with its size and effort set by difficulty.

Where does the trusted IP list come from?

From your EDR/RMM. SentinelOne knows the public IP of every endpoint running its agent, and Sentra turns it into the allowlist and keeps it current as the fleet changes. It is not a list you maintain by hand, and the IP is observed by the EDR on the agent's connection, not self-reported by the machine.

What about an operator working remotely, roaming or from CI?

Access is bound to your endpoints' IPs, so a laptop with the agent on a new IP gets in once the inventory picks it up, and for roaming or CI runners you start in monitor mode before blocking. It is a network control, with honest limits (an office behind one NAT is a single address), not magic.

Do my prompts or keys pass through your servers?

No. The gateway runs in your infrastructure, against your Azure account and your own Cloudflare one. The provider key never leaves the gate, and the decision model only sees the last trimmed message to pick the model, never your tools or your system prompt.

Does it lock me into one provider or one client?

No. The gate translates the Responses API to Anthropic's and to the other models on the resource, so the same deployment serves several models without touching your agent. If you switch model or resource tomorrow, you change the gate's config, not your workflow.

An offensive-AI gateway that won't bankrupt you.