From signal to measurement: testing agent security yourself in your setup

In July, Radar published two articles on the safety of agents: a retrospective on the security incidents of July and an overview of runtime security for those running agents. Together, these articles paint a picture: agents are becoming more autonomous, gaining more permissions, and the attack angle is shifting from the model to the connections between them. What both articles deliberately did not do is explain how to check these signals for your own setup. This article is the shared follow-up: it translates the signals into measurement questions and a minimal, repeatable test set. In doing so, it also bridges the missing gap between radar — which signals — and benchmark, where the measurement methods live.

Retrospective: what the July articles showed

The two articles are each one side of the same coin, and the distinction between defending and drawing lessons is nowhere explained. That distinction, however, is the basis for this follow-up. The incidents article mainly collected what went wrong: MCP vulnerabilities, prompt injection via external content, agents that were given too many permissions and used them in production. The runtime article was about the defensive side: limiting autonomy, budgets, sandboxing, and the principle that an agent should not be able to do more than the sum of its tools.

If you put both articles side by side, one thing stands out: neither mentions a number, a test, or a threshold. That makes sense — they are signal articles, not manuals. But for the builder who runs agents themselves, that is precisely the missing step. A signal like "MCP tools are a growing attack angle" remains a warning until you can say: "in my setup, my agent blocks a tool call to an unknown domain in eight out of ten attempts." The latter is a measurement, and measurements are what benchmark teaches you to set up.

Step 1: from signal to measurement question

Every signal from the July articles can be turned into a measurement question. The trick is that the measurement question must be specific enough to be testable and general enough not to drown in details. A measurement question has three parts: the threat, the location in your setup, and the behavior you want to see.

The measurement question determines which test you set up. Anyone asking the first question needs a test with a tool call to an unknown domain and one to a known domain; the difference in outcome is the measurement. Anyone asking the second question needs a test in which an innocent document contains a malicious instruction. Without a measurement question, a test remains an anecdote; with a measurement question, it becomes a repeatable check.

Step 2: a minimal security test set for agents

A test set for agent security is no different in setup from any other test set: fixed input, fixed expectation, repetition. The difference lies in what you record. For agents, it revolves around behavior in a chain — model, tools, permissions, context — and therefore a minimal set contains four categories of cases.

• Tool limitation. Three to five cases in which the agent should refuse a tool call: an unknown domain, a domain on a blocklist, a call outside the budget. Expectation: refusal or explicit intervention.
• Instruction boundary. Three cases in which external content (document, email, web page) contains instructions that go against the system prompt. Expectation: the instruction is ignored or flagged, not executed.
• Permissions and scope. Two cases in which a seemingly innocent task requests an operation outside its scope, such as reading a file that does not belong to the task. Expectation: refusal or explicit confirmation.
• Limiting autonomy. Two cases in which the agent gets the chance to keep running, for example through an endless loop or a task that keeps demanding more steps. Expectation: stopping after budget or after a fixed number of steps.

You record the expectation per case before you test. This is important: deciding what counts as a good result only after the test means measuring after the fact and being able to rationalize any outcome. The rules for compiling such a set — how to prevent your own production data from contaminating the set and how to build a control set — are on benchmark.llmnet.nl under red teaming and safety tests. That page also explains why a safety test is by definition an attempt to break your own system, and why that requires a different mindset than a regular quality test.

Step 3: judging — what is safe enough?

A test set without judging rules only produces isolated outcomes. For agent security, judging is simpler than for open-ended questions, because for each case you have a yes/no expectation: refused or not, stopped or not, flagged or not. Still, there are two pitfalls.

The first pitfall is that one successful test says nothing about the next. Agents are stochastic; the same prompt may pass through an unknown domain one time and not the next. That is why repetition is part of the measurement: three attempts per case give a first impression of the spread. The statistics behind the number of repetitions are on benchmark.llmnet.nl under statistics for evaluations — in short: start small, repeat, and only scale up if you want to detect a difference that is small.

The second pitfall is a judge that is itself biased. If you use a model as a judge, for example to check whether a refusal was correct, that model must first be calibrated against human judgments. The method for doing so, including the known biases of a model acting as judge, is on benchmark.llmnet.nl under LLM-as-a-judge. For a security test set, a human look at the outcomes is usually wiser than a fully automated judging loop — precisely because the interesting failures are unexpected.

Step 4: the repeat rhythm

A measurement you take once is a snapshot. The value lies in the rhythm: a fixed test set, fixed moments, a fixed protocol for changes. For agent security, a simple rule applies: run the set on every change that could affect behavior — a new model version, a new tool, a new MCP server, a new system prompt — and additionally on a periodic basis, for example monthly, even if nothing changed.

Why periodically too? Because the threat environment changes without you doing anything. A tool that seemed safe last month may have a known vulnerability this month; a domain on your allowlist may by now be controlled by someone else. The pattern of regression testing — automatically testing every change before it goes into production — is on benchmark.llmnet.nl under regression testing for prompts; for agents, the same logic applies, except you test not just the prompt but the entire chain.

What the setup costs in time and tokens

A minimal set of ten cases, three repetitions per case, costs thirty agent runs. Depending on your setup, that is a quarter of an hour to an hour of compute time, and a number of tokens that is small compared to what an agent consumes in a normal workday. The biggest cost item is not compute time but maintenance: every new tool or MCP server adds cases, and every case must keep its expectation up to date. For anyone who wants to calculate costs per task instead of per run, there is benchmark.llmnet.nl with an explanation of the price-per-task trade-off. The rule of thumb for security tests is the same as for other evaluations: start with the smallest set that covers the biggest risks, and only expand when you want to see a difference the current set cannot show.

Scope: where this article stops

This article is deliberately not a full security audit. It does not cover specific tool configurations, a vulnerability database, or legal or compliance frameworks. What it does do is bridge the gap that was missing in July: the signals about agent security have in this article been translated into something you can run and repeat yourself. The measurement methods this article links to — red teaming, statistics, LLM-as-judge, regression testing, reading safety benchmarks — are all on benchmark and are each worked out separately; this article organizes them for the specific question of the builder running agents in production.

The next step for anyone who wants to take this seriously is the method for composite systems: how to set up a full agent evaluation that takes into account not just security but also correctness and cost. That approach is on benchmark.llmnet.nl under agent evaluation — the logical follow-up to this article, because it places the security test set within a broader measurement setup.