AI pentest guardrails: five controls, and how to check each one

Arxiis runs every authorized penetration test through five safety controls. Each one is a check that sits outside the model, not an instruction inside it. This post takes them in the order a run meets them: authorization file, scope guard, approval gate, evidence isolation and watchdog. For each, we say what it does, what it does not do, and the question we would ask if we were the buyer.

Why a prompt is not a guardrail

Many AI pentest guardrails are written into the agent's instructions. "Stay in scope." "Do not touch production." "Ask before escalating." Those sentences shape what the model tends to do. They do not bind what it can do.

A model can misread a hostname, follow a redirect, or decide that "this subdomain is obviously part of the target". It will then act on that reasoning. The instruction to stay in scope sits in the same context window as the reasoning that broke it. Nothing outside the model is there to say no.

OWASP describes this failure mode for LLM applications in general. Its 2025 Top 10 for LLM Applications lists Excessive Agency as LLM06. Its mitigations include human approval for high-impact actions, and doing authorization in downstream systems rather than relying on the LLM to decide.

That is the principle behind every control below. The model proposes. Code and named people decide. Autonomous pentesting safety comes down to one question: when the model is wrong, what stops it? The five controls are our answer.

1. The authorization file

Arxiis will not touch a non-local target without a written SCOPE + ROE authorization file. No file, no engagement.

The file names the authorizing officer, the in-scope hosts, the methods allowed on each, and the exclusions. It also sets the testing window, forbidden techniques, escalation approver, stop conditions, budget and evidence retention period. Our rules of engagement template covers every field.

The file matters because the system reads it, not just the people who sign it. NIST's glossary, drawing on SP 800-115, describes rules of engagement as detailed guidelines and constraints for a security test. They are set before the test starts. In most engagements those constraints live in a PDF the tester has read. In ours they are inputs that the other four controls enforce.

The exception is a local target. A lab application on the operator's own machine needs no file. Anything else does.

What to ask a vendor: "Start a run against a non-local target with no authorization file, or an unsigned one. What happens?" The answer you want is a refusal to start, not a warning.

2. The scope guard

The scope guard reads the authorization file and checks every outbound request against it before the request leaves. A host that is not in scope is blocked. A host listed for enumeration only gets nothing beyond enumeration.

The check is deterministic. The model cannot add a host, widen a wildcard, or argue the guard out of a block.

When the guard blocks a request to an out-of-scope host, the run halts and reports. It does not drop the request quietly and carry on. A blocked request means the crew's picture of the target has drifted from the file. That is a moment for a human to look.

Scope changes go through the file, never around it. An operator who needs a new host amends the authorization, gets the change approved in writing, and the guard reads the new version.

What to ask a vendor: "Is scope enforced in code on every request, or is it an instruction to the model? After a block, does the run halt or continue?" Then ask to see a blocked request in the pilot logs.

3. The approval gate

Some steps are not the crew's to take. Moving from one tier to another, or doing anything beyond enumeration on a host marked enumeration-only, raises an approval request in the operator console. The run waits.

A named human reads what the crew wants to do and why, and approves or declines. The decision and the person who made it are written to the execution ledger. The system cannot approve its own next step. There is no setting that lets it, and no confidence threshold above which it skips the human. Our post on human-in-the-loop AI pentesting makes the case for keeping that decision human.

If nobody answers, the safe default is that the run waits and then halts and reports. Silence is never read as a yes.

What to ask a vendor: "Which actions need human approval? Can the agent ever approve its own escalation? Where is each approval recorded, and does the record carry the approver's identity?"

4. Per-engagement evidence isolation

Every engagement's evidence stays inside that engagement. The requests, responses and ledger records from one client's test are kept apart from every other engagement.

This is a safety control as much as a privacy one. Evidence often contains data from the target environment. A tool that pools evidence across engagements builds a store of other people's data. It also opens a path for one client's details to surface in another client's report. Isolation closes both.

It also keeps findings honest. A finding is labelled proven only when its evidence binds to a real ledger record; otherwise it is labelled needs review. Isolation means that record has to come from this engagement, run against this scope, inside this window.

What to ask a vendor: "Where is my evidence stored, and who can read it? Is it ever used across customers, including to tune a model? How is it deleted when the retention period ends?"

5. The budget and time watchdog

The watchdog enforces the window and the budget set in the authorization file. When time or budget runs out, the run stops. It does not round up.

It then writes a partial report. The report says the run stopped early, what was covered and what was not, and labels every finding proven or needs review.

The failure mode it prevents is the false completion. That is a report that looks finished because the tool ran out of road, not out of targets. A partial report that says so is worth more than a full-looking one that hides the gap. Our post on fail-closed design takes that argument further.

What to ask a vendor: "What does the report say when a run hits its time or budget limit? Show me one." If the sample report has no way to say "stopped early", assume it never does.

How the five fit together

Read in order, the controls form a chain. The file defines what is allowed. The scope guard enforces it on every request. The approval gate holds the decisions that need a person. Isolation keeps the evidence inside the engagement it came from. The watchdog ends the run honestly when the limits are reached.

Every command the crew runs, and every approval or decline, lands in the tamper-evident execution ledger. A scope halt or an early stop is stated in the report. That is what turns a list of safety features into something a client or an auditor can check afterwards. Our post on penetration testing evidence covers what the records contain and how to hand them over.

A pilot checklist for safe AI pentesting

Guardrails are claims until you test them. Bring these six checks to any pilot, ours included.

1. Start a run with no authorization file. Does it refuse?
2. Leave a known out-of-scope host within reach. Is the request blocked, and does the run halt?
3. Reach a step that needs approval. Does the run wait, and is the approver recorded?
4. Ask where the evidence lives and who else can read it.
5. Set a short budget. Read the report it produces when the budget runs out.
6. Ask for the records behind each of the five checks above.

A vendor whose guardrails are enforced in code can show all six in an afternoon. A vendor whose guardrails live in a prompt will have to explain why the test does not apply.

What these guardrails do not do

They enforce the file. They do not check that the file is right. If the authorization lists a host the client does not own, the scope guard will faithfully allow it. Confirming ownership is a human job before anyone signs.

The approval gate is only as good as the person answering it. An operator who approves every request without reading has turned the gate into a delay. Track how often requests are declined; a decline rate of zero is worth a conversation.

The ledger is tamper-evident, and only tamper-evident. The chain is unkeyed. It exposes a quiet edit to one record. It does not stop a party with write access from rewriting the chain from that point on. Store the final hash somewhere your own team controls.

And the controls cost capability. A run that halts on a scope block, waits for approval or stops at its budget covers less ground. A run with none of those controls covers more. On a benchmark there is no scope to leave and nothing to approve, so that cost is invisible. On someone else's production systems, it is the point.

Frequently asked questions

What are AI pentest guardrails?

Controls that limit what an AI-driven penetration test can do, enforced outside the model. Arxiis has five. They are a SCOPE + ROE authorization file, a scope guard, a human approval gate, per-engagement evidence isolation, and a budget and time watchdog.

Can the AI in Arxiis expand its own scope?

No. Scope comes only from the authorization file, and the scope guard checks every outbound request against it in code. Changing scope means amending the file and getting the change approved in writing.

What happens when an Arxiis run hits its time or budget limit?

The watchdog stops the run and writes a partial report. The report says it stopped early and what was not covered, and labels every finding proven or needs review. It does not present the run as complete.

Is a prompt instruction enough to keep an AI pentest in scope?

No. An instruction in the prompt shapes what the model tends to do but does not bind what it can do. Scope and approval should be enforced by deterministic checks and named people outside the model.

What should I ask an AI pentesting vendor about safety?

Five things. Does it refuse to start without an authorization file? Is scope enforced in code on every request? Can the agent approve its own escalation? Where is evidence stored, and who can read it? What does the report say when a run stops early?