Run it¶
Requirements¶
- Docker with Compose v2,
make, and uv (it fetches Python 3.12 itself). - About 10 GB of free disk for option A (local model), about 5 GB for option B; 16 GB of RAM recommended.
Two ways to run the models¶
The gateway, the controls and the demo are the same in both.
qwen3:8b on Ollama in Docker. No accounts and no paid APIs; nothing leaves the machine.
On a CPU-only laptop a model turn can take up to a minute.
Model and AI-judge calls go to deepseek/deepseek-v4.1-flash on OpenRouter's
zero-data-retention endpoints only, so turns take seconds. Needs an OpenRouter API key.
Personal data is masked, and secrets and injections are blocked, before anything leaves the
machine; the policy pins zero retention and a price ceiling the agent cannot override.
Details: OpenRouter mode.
1 · Get the code and create .env¶
git clone https://github.com/JMadejczyk/ai-security-gateway.git
cd ai-security-gateway
python3 -c "import re,secrets;s=open('.env.example').read();open('.env','w').write(re.sub(r'=change-me\S*',lambda m:'='+secrets.token_urlsafe(32),s))"
That replaces every placeholder secret with a random one and makes the file readable only by you. For option B, also add your key:
Ports already in use?
If 9090, 8080 or 3300 is taken, add ACL_OPERATOR_HOST_PORT, ACL_AGENT_HOST_PORT or
ACL_GRAFANA_HOST_PORT to .env now (for example ACL_OPERATOR_HOST_PORT=19090), and use
that port in the commands below.
2 · One-time downloads¶
docker compose --profile init run --rm models-init # injection classifier, ~740 MB: both modes
docker compose --profile init run --rm ollama-init # local model, ~5 GB: option A only
Each can take several minutes, depending on your connection.
The gateway refuses to start with prompt_injection enabled and no verified classifier.
3 · Start the stack¶
The first start builds the Docker images, so allow a few minutes; later starts take seconds.
| What | Where |
|---|---|
Agent API (models /v1, tools /mcp) |
http://127.0.0.1:8080 |
| Operator API | http://127.0.0.1:9090 (/healthz shows "llm_upstream": "local" or "remote") |
| Grafana | http://127.0.0.1:3300, user admin, password ACL_GRAFANA_ADMIN_PASSWORD from .env |
From B back to A: make demo-off REMOTE=1, then make demo-up.
4 · See it work¶
make demo # 7 narrated scenes; exits 1 on any deviation; ~40 s in option B
make smoke # mixed traffic, so the Grafana dashboards fill up
The scenes: the same question from two people, an injection that taints the session, an autonomous job held for human approval, a too-expensive SQL query, PII masked and secrets blocked, no way around the gateway, and a live policy change. See the demo walkthrough.
5 · Run the test suite¶
make test # ~2,990 tests in about 75 s + a check that all 15 controls have passing allow AND deny tests
make models # optional: fetch the classifier locally so the 66 real-model tests run too (3,034 in total)
No Docker and no API key needed. More in Test suite.
6 · Try your own prompts¶
Replace 9090 if you changed ACL_OPERATOR_HOST_PORT.
TOKEN=$(curl -s -X POST 127.0.0.1:9090/auth/demo-token -H 'content-type: application/json' \
-d '{"sub":"anna@demo"}' | python3 -c "import sys,json;print(json.load(sys.stdin)['access_token'])")
curl -s 127.0.0.1:8080/v1/chat/completions -H "authorization: Bearer $TOKEN" \
-H 'content-type: application/json' \
-d '{"model":"deepseek-v4.1-flash","messages":[{"role":"user","content":"My AWS key is AKIAQ3EGRVW6XKZT4M7N, which region?"}]}'
# {"error":{"message":"blocked by policy: secret_detected", ...}}
The model is deepseek-v4.1-flash in option B and qwen3:8b in option A;
GET /v1/models with the token lists what is served. Demo identities: anna@demo (analyst)
and bartek@demo (intern). Any OpenAI-compatible client works against /v1 with that token,
and make opencode AS=anna runs a real coding agent through the gateway
(guide).
7 · Change the rules live¶
Edit config/policy.yaml (controls, thresholds, block vs redact, allowed models, budgets, the
OpenRouter settings) or config/feeds/signatures.json (the attack-signature feed). The gateway
hot-reloads within seconds, records the new revision in the audit log and marks it on the dashboards. An
invalid edit is rejected and the last valid policy stays in force. See Policy file.
Why an LLM?¶
- It's the model the agents use. The gateway is a proxy in front of it, so the demo needs a real model for opencode and DataBot to reason and call tools.
- It powers the AI judges, the semantic half of the hybrid defense. Rules catch patterns (keys, PESEL, SQL cost, known attack signatures). A small LLM judges what depends on meaning: paraphrased prompt injections, hidden instructions in MCP tool descriptions, tool calls that don't fit the user's request (held for approval), and leaks in the model's answers (masked).
Cheap first: rules on every call, a local non-LLM classifier on inputs, the LLM only for
uncertain cases. A judge that fails or times out fails closed. Without the judges: section
in config/policy.yaml the AI checks turn off and the rules keep working. make test needs no
LLM.