Fail-closed by default: why the safe choice is to stop

When an Arxiis control cannot reach a clear yes, the answer is no. No authorization file means no run. A host missing from the scope file means a halt. An escalation nobody has approved means a wait. A spent budget means a stop and a partial report. A claim the evidence cannot carry means the words leave the report.

This post explains that design choice, control by control, and what it costs.

Two ways for a control to fail

Every control in an autonomous system meets moments when it gets no clean answer. The input is missing, ambiguous or late. Or the check itself errors. What happens next is the design choice that matters.

A fail-open control lets the action through. A fail-closed control blocks it. The NIST glossary takes its definition of "fail secure" from CNSSI 4009. It describes a mode that keeps a system in a secure state when a failure occurs or is detected.

The idea is old. Saltzer and Schroeder's 1975 paper, "The Protection of Information in Computer Systems", called it fail-safe defaults: base access decisions on permission rather than exclusion. Their reasoning still holds. A mechanism that grants explicit permission tends to fail by refusing, and a refusal gets noticed quickly. A mechanism that lists exclusions tends to fail by allowing, and that can go unnoticed for a long time.

"Fail-safe" depends on what you protect

The vocabulary is slippery. In physical access control, a fail-safe lock unlocks when power is lost, so people can leave the building. A fail-secure lock stays locked. "Safe" there means safe for the people inside.

So before choosing a default, name what you are protecting. For an AI pentesting tool, we protect two things. The first is every system outside the client's written authorization. The second is the truth of the report. Stopping protects both. In our system, fail-safe and fail-closed point the same way.

What fail-open looks like in an AI pentester

A language model is a continuation engine. Left to itself, it continues.

Faced with an unclear scope, it reasons that a neighbouring host is probably in scope. Faced with an escalation decision, it reasons that the next step is obviously justified. Faced with a spent budget, it writes a summary that reads as finished. Faced with an unconfirmed lead, it reports the lead with a severity.

None of that needs bad intent. Each is the model producing a plausible next step, which is what it is built to do. The failure is architectural. Nothing outside the model had the final word.

A prompt instruction does not fix this. "Stay in scope" in a system prompt is a request, and requests get reasoned around. A fail-closed control has to sit outside the model and must not take the model's reasoning as input.

Five controls, one default

Each Arxiis control is either deterministic or human. Each has a defined answer for the case where it cannot decide, and that answer is always the conservative one. Our post on AI pentest guardrails walks through all five end to end. Here the focus is the default.

Authorization: no file, no run. Arxiis will not engage a non-local target without a written SCOPE + ROE authorization file. There is no reduced mode for a non-local target without one. The run does not start. Our rules of engagement template covers what the file should contain.

Scope guard: unlisted means out. The scope guard checks every outbound request against the authorization file. The file is an allow-list. A host that is not on it is out of scope, however closely it seems related to one that is. When the guard blocks, the run halts. The model cannot argue past the guard, because the guard never reads the model's argument.

Approval gate: silence is not consent. Escalation needs a named human. When the crew wants to go further, it raises a request and the run waits. An unanswered request never turns into a yes. If the wait ends, it ends in a stop. Each decision, and the operator who made it, is recorded in the ledger.

Watchdog: out of budget means stop and say so. The watchdog enforces the time and budget limits set for the run. When a limit is reached, the run stops and writes a partial report. Partial means honest about being partial: the report does not present itself as complete, and every finding keeps its proven or needs review label. A false completion would be easier to read. It would also be wrong.

Report scrubbing: unsupported means unsaid. The last control works on language. A finding is labelled proven only when its evidence binds to a real record in the tamper-evident, hash-chained execution ledger. Everything else is needs review. Before the report leaves the system, it is deterministically scrubbed of claims the evidence does not support. Our post on why we let our AI say "I don't know" covers the thinking behind that label.

One more boundary works the same way. Evidence is isolated per engagement. Findings and records from one engagement are not available to another. The default for cross-engagement data is no access.

What the client receives after an early stop

A run that ends on the watchdog, a scope halt or an unanswered approval still produces a report. In it, the client finds:

• a clear statement that the run stopped before it finished
• findings labelled proven, each bound to a ledger record
• findings labelled needs review, kept apart from the proven ones
• language limited to what the evidence supports

The report does not fill gaps with what the run would probably have found. A partial, honest report tells the client exactly where to point the next run. A complete-looking report that guesses in its last section tells them nothing they can rely on.

What fail-closed does not solve

A stop limits what happens next. It does not undo what already happened. Whatever the run had done on in-scope systems before the stop is still done. Plan test windows and blackouts as if any run might stop at any moment.

The scope guard enforces the file you wrote, including its mistakes. If someone lists a production host that should have been excluded, the guard faithfully allows it. Fail-closed moves the main risk into the authorization document. That is the right place for it, because a human can read and sign that document. But a human has to read it.

The authorization rule applies to non-local targets. Local lab targets run without a file, which suits benchmarking and practice. It also means "no file, no run" is a promise about other people's systems, not about a laptop.

The honest cost

Failing closed is not free, and the costs show up in places buyers notice.

A lower headline number. On XBOW-104 (19 October 2026) Arxiis solves 56% raw and 44% confirmed. We lead with the confirmed figure. On hard chained and blind attacks we solve 0–42%, a known weak spot that failing closed does nothing to hide.

Runs that halt on paperwork. A legitimate host left off the scope file blocks the run. The fix is a human edit and a re-sign. That is friction, and it is deliberate.

Runs that wait. Approvals need someone available. A run with no operator to answer makes no progress.

Coverage left for later. A watchdog stop leaves part of the scope untested. The client has to schedule another run to cover it.

Fewer findings on the page. A fail-open tool will show more. Some of its extra findings will be real, and some will not.

How to test which kind of tool you are buying

You can see a tool's defaults in a short pilot. Run five drills against a non-local staging system you are authorized to test:

1. Start a run with no authorization file. It should not start.
2. Give it an authorized scope with a reachable host outside it. It should halt, and the attempt should be logged.
3. Leave an approval request unanswered. It should never proceed on silence.
4. Set a very small budget. It should stop and say the report is partial.
5. Read the wording of an unconfirmed lead. It should claim no impact.

Our evaluation scorecard puts these alongside the other questions worth asking, and the safety page sets out how we handle each one.

A tool that passes all five will report less than one that fails them. Whether that is a cost or the product depends on whether you need to act on what the report says.

Frequently asked questions

What is the difference between fail closed and fail open?

When a control cannot reach a clear decision, a fail-open control lets the action through and a fail-closed control blocks it. In an AI pentesting tool, fail-open means the agent keeps going and reports when unsure. Fail-closed means it stops, waits or labels the result as unconfirmed.

Is fail-safe the same as fail-closed?

It depends on what is being protected. In physical access control a fail-safe lock unlocks on power loss so people can leave. For an AI pentesting tool, the things to protect are systems outside the authorization and the accuracy of the report. For both, the safe failure is to stop.

What does Arxiis do when its time or budget runs out?

The watchdog stops the run and Arxiis writes a partial report. The report says the run ended early, keeps every finding labelled proven or needs review, and does not present itself as complete.

What happens if an Arxiis run reaches a host that is not in scope?

The scope guard checks every outbound request against the written authorization file, which works as an allow-list. A host that is not listed is treated as out of scope, the request is blocked and the run halts.

Does failing closed lower Arxiis's solve rate?

It lowers the headline number. On XBOW-104 Arxiis solves 56% raw and 44% confirmed, and leads with the confirmed figure. Hard chained and blind attacks, at 0–42%, are a known weak spot.