0% false positives isn't a feature. It's an architecture.

Arxiis has a measured false-positive rate of 0% on XBOW-104, run dated 19 October 2026. That is not because the language model never hallucinates. It hallucinates like every other model. It is because the system is not allowed to call anything proven unless a deterministic check agrees.

The failure mode in systems that trust the model

An LLM-driven pentesting agent produces a stream of claims. "This parameter is injectable." "This endpoint leaks a token." "This template renders user input." Most AI pentesting tools do one of two things with those claims. They report them as findings, with a severity the model chose. Or they ask the model a second time, "are you sure?", and report the ones that survive.

Both approaches put the model in charge of its own evidence. A model that is wrong the first time is often wrong the second time, with more confidence. The result is a report where a real SQLi sits next to a hallucinated one with the same severity label, and the security team has to re-test both by hand. That re-testing is the cost of AI pentest false positives, and it is usually larger than the time the tool saved.

The fix is not a better model. The fix is to take the decision away from the model.

The proof spine

Arxiis bolts the agent crew to a deterministic proof spine. It has three parts.

The execution ledger. Every command the crew runs is written to an append-only ledger. Each record carries a sequence number, a timestamp, the command, the observed response, and a hash that includes the previous record's hash. Change one record, and every hash after it stops matching. The ledger is tamper-evident. It is tamper-evident only; the chain is unkeyed, so a party with write access could rewrite the whole chain from the edit onward. What they cannot do is edit one line quietly.

The binding rule. A finding is a claim plus an evidence pointer. The pointer must resolve to a ledger record whose observed response actually contains the evidence the finding cites. If it does, the finding is labelled proven. If the pointer is missing, or the record does not contain what the finding says it contains, the label is needs review. There is no third state, and the model does not choose the label. A function does.

label(finding):
    rec = ledger.get(finding.evidence_ref)
    if rec is None:                       return NEEDS_REVIEW
    if not matches(rec.response, finding.evidence_pattern):
                                          return NEEDS_REVIEW
    if rec.hash != ledger.recompute(rec): return NEEDS_REVIEW
    return PROVEN

The report scrubber. Before a report leaves the system, every sentence is checked against the labels. Impact language attached to a needs review finding is removed. Phrases that claim more than the ledger shows, "exposes all customer data", "full database compromise", are cut even on proven findings unless a ledger record shows that specific thing. The report says what was observed, and nothing else.

Signal confirmation

The binding rule only works if the evidence is real. For a large class of bugs the response alone is not enough: a blind SQLi returns the same page either way, a blind SSRF gives no body, an SSTI may evaluate without reflecting.

For those, the crew runs a signal-confirmation step before the finding is even a candidate. Out-of-band callbacks to a listener the system controls. Differential timing across controlled variants. Reflected canaries with unique nonces. The confirmation request and its observation are ledger records too, so the binding rule can point at them.

When confirmation is impossible in the target environment, the finding stays needs review. That is the honest answer, and it is also why some correct exploits never become confirmed solves on the benchmark.

What this costs

Zero false positives is a choice with a price, and the price is visible in our numbers. On XBOW-104 we solve 56% raw and 44% confirmed. The 12-point gap is mostly findings that were right but could not be bound to evidence. A system willing to report them would show a higher number and a non-zero false-positive rate. We think that is the wrong trade for a tool whose output a human has to act on.

The other cost is engineering. Every new executor needs a confirmation strategy before it ships, and some vulnerability classes are far harder to prove than to find. SSTI is the worst of them, and it gets its own post.

What it does not solve

The proof spine proves that a specific request produced a specific response. It does not prove business impact, lateral movement, or what an attacker would do next. We do not claim any of those. It also does not help with the bugs the crew never finds, and on hard chained attacks the crew finds between 0% and 42%, which is a known weak spot.

So the architecture guarantees one thing: when the report says proven, it is. Whether that is worth a lower headline solve rate is the question every buyer has to answer for themselves. We have answered it for us.

What a false positive costs your team

A false positive is not a harmless extra line in a report. It consumes engineering time, and it erodes trust in every finding that follows. Three costs are worth naming.

Triage time. Each unverified finding needs someone to read it, reproduce it and decide whether it is real. The more findings a report holds, the larger the share of a senior engineer's week that goes to answering "is this true?" instead of fixing things.

Fix-queue pollution. A false positive that reaches a developer's backlog wastes their time again. They investigate, cannot reproduce it, and close the ticket. Do that a few times and developers start treating security tickets with suspicion.

Credibility loss. A report that has been wrong once gets read more skeptically afterwards. Real findings then take longer to be believed and acted on.

None of these costs shows up in a licence price. They are the reason evidence per finding matters more than raw finding count.

How to audit a false-positive claim

Any vendor can write "low false positives" on a slide. Here is how to check a specific claim.

1. Ask for the number and the method. "0% measured on XBOW-104, from an evidence audit of every confirmed solve" is a claim you can examine. "Near zero" is not.
2. Ask what was counted. Was the rate computed over confirmed findings only, or over everything the tool reported? The two give very different answers.
3. Ask who checked. An independent re-observation by a second process is stronger than the same system grading its own work.
4. Ask to see a sample. Pick three findings from the vendor's sample report and ask for the stored evidence behind each one.
5. Run it on your own target. During a pilot, have your engineers review every confirmed finding and record how many they agree with.

For Arxiis, the first four are answerable from the benchmark audit and the ledger, and the fifth is what the pilot is for.

How proven and needs review fit into your workflow

Labels only help if your team's tooling respects them. A simple setup works for most teams.

proven findings go straight to the owning team's backlog, with the ledger record attached. The developer opens the ticket and sees the exact request and response that demonstrate the issue. No separate verification step is needed before work starts.

needs review findings go to a short triage queue owned by a security engineer. They are not tickets for developers yet. The engineer decides whether to investigate further, re-run the confirmation step with a different strategy, or close the finding.

This split is why the labels matter. It keeps the developer backlog clean and puts the uncertainty where a specialist can resolve it.

Evidence retention and audit

Because every command and response is a record in the tamper-evident ledger, the evidence behind a finding can be kept as long as your policy requires. Auditors reviewing a test can ask for the record behind any finding and re-check that the chain is intact.

Two practical points. Agree a retention period in the authorization file, since evidence may contain data from the target environment. And remember the ledger is tamper-evident: it shows if a record was edited, so store the final hash somewhere your own team controls.

The limit of the claim

A 0% measured false-positive rate on XBOW-104 describes the confirmed solves on that benchmark. It does not predict a rate on your estate, and it says nothing about issues the tool did not find. Use it as a measured starting point, then confirm it on your own pilot.

Frequently asked questions

Why do AI pentesting tools produce false positives?

Most tools let the language model decide what counts as a finding and how severe it is. Models hallucinate, and asking the model to double-check itself does not fix that. Arxiis removes the decision from the model: a finding is labelled proven only when a deterministic check binds its evidence to an execution ledger record.

What does 'needs review' mean in an Arxiis report?

The finding has no ledger record that contains the evidence it cites, so Arxiis will not call it proven. A human reviews it. It is the only other label; there is no in-between state.

Can the Arxiis execution ledger be edited without anyone noticing?

No. It is tamper-evident. Records are hash-chained, so editing one record breaks every hash after it. The chain is unkeyed, so it does not stop a party with write access from rewriting the chain from that point on; it makes a quiet single edit impossible.

What does a false positive cost a security team?

Triage time to reproduce and decide, wasted developer time when a false finding reaches the backlog, and lost credibility that slows action on real findings.

How should proven and needs review findings be handled?

Proven findings go to the owning team's backlog with the ledger record attached. Needs review findings go to a short triage queue owned by a security engineer, who decides whether to investigate, re-run confirmation, or close them.