Why we let our AI say "I don't know"

The most important output Arxiis produces is not a finding. It is the label needs review, which is how our system says "I don't know" instead of guessing. We built it as a first-class output, with its own queue and its own section of the report. An AI pentester that cannot admit uncertainty will, sooner or later, hand a client a hallucinated finding with a severity attached.

This is the case for that decision, and an account of what it costs us.

The case for always having an answer

The opposing view is reasonable, and it is the one the market rewards. Buyers compare tools on a headline number. A tool that reports every plausible lead looks thorough. A tool that files half its leads under "we could not confirm this" looks timid. On a sales call, the confident report wins.

There is a technical version of the argument too. Language models are good at spotting the shape of a bug. If the model is usually right, throwing away its unconfirmed hunches throws away signal. Let the client's engineers sort it out. They know their systems better than any tool does.

And there is a commercial version. Admitting uncertainty is a sales risk. A competitor that reports more will look better in a side-by-side. Most buyers never get as far as the evidence audit that would show the difference.

I take all three seriously. I still think they are wrong for this job.

Why models guess

AI hallucinated findings are not a bug in one vendor's prompt. They come from how language models are trained and scored. In September 2025, OpenAI researchers argued in a paper titled "Why Language Models Hallucinate" that training and evaluation reward guessing over admitting uncertainty. A model that guesses sometimes scores. A model that abstains never does. So models learn to guess.

Put that model inside a pentest loop and the habit comes with it. Asked whether a parameter is injectable, it would rather say yes with a plausible story than say it cannot tell. The story usually cites something real: an error message, a timing wobble, a reflected string. Each can be a true observation attached to a false conclusion.

We cannot train that out of the model. Arxiis is model-agnostic, and the flagship is validated on Claude Sonnet. Whatever model sits in the crew, we assume it will sometimes be confidently wrong. The only question is what the system around it does next.

A wrong finding is a trust event

In most software, a wrong answer costs a retry. In a penetration test, a wrong finding costs belief.

A client receives a report with a critical SQLi. An engineer spends a day trying to reproduce it and cannot. From that moment, every other finding in the report is read with suspicion, including the real ones. The next report from the same tool gets the same treatment. One unsupported critical can undo an engagement's worth of correct work.

Open-source maintainers have seen this from the receiving end. In January 2026 the curl project said it would end its HackerOne bug bounty. The trigger was a stream of low-effort reports, many apparently AI-generated, that turned out not to describe real vulnerabilities. The stated aim was to remove the incentive to send poorly researched reports. When unverified claims are cheap to produce, the cost lands on whoever has to check them.

A pentest client is in that maintainer's seat. The difference is that the client paid for the report.

What "I don't know" looks like in our system

needs review is not a softer severity. It is not a footnote, and it is not a confidence score. It is one of exactly two labels a finding can carry, and the model does not choose between them.

Every command the crew runs is written to a tamper-evident, hash-chained execution ledger. A finding is labelled proven only when its evidence binds to a real record in that ledger, one that shows what the finding claims. If it does not bind, the label is needs review. A deterministic check makes that call. The model's own confidence plays no part in it. The mechanics are in our post on AI pentest false positives and on the proof spine page.

Then the report scrubber runs. Report language is deterministically cleaned of claims the evidence does not support. Impact language attached to a needs review finding does not survive. The model can draft the prose. It cannot draft certainty it has not earned.

So the client gets two honest lists: what we can show, and what we saw signs of but could not show. Both are useful. Only the first should go straight to a developer's backlog. The second goes to a person, which our post on human-in-the-loop pentesting covers in detail.

Why not a confidence score?

We are often asked why we do not attach a percentage to each finding instead. Three reasons.

A confidence score is the model grading its own work. That is the exact thing we are trying to remove from the decision.

A score invites a threshold, and thresholds get tuned to make the report look right. Move the cut-off and the finding count changes, while nothing about the target has changed.

And a score does not tell the reader what to do. "0.7" leaves the engineer guessing. needs review says what happens next: a human looks at the stored request and response before anyone acts on it.

What it buys

On XBOW-104 sweeps (19 October 2026), our measured false-positive rate is 0%. Every confirmed solve was independently re-observed. That number exists because the system is allowed to decline. A tool that must always answer cannot have it.

The same discipline is why we lead with 44% confirmed rather than 56% raw on that benchmark. The raw figure counts what the crew observed. The confirmed figure counts what we could bind to evidence. We also publish our known weak spot: hard chained and blind attacks, where we solve 0–42% depending on sub-tier. The full breakdown is in our XBOW-104 results post.

The concession

needs review is overhead, and we hand it to a human. Someone has to look at each item: read the stored evidence, decide whether it deserves a manual check, then close it or re-run it. A tool that reported everything as a finding would hand over the same work, disguised as results. Ours hands it over labelled. It is still work, and on a large scope it is real hours.

It also costs us in comparisons. Some of what sits in needs review is real. A tool willing to report those leads as findings will show more findings and a higher headline rate. On a short evaluation, the buyer may never reach the evidence that separates the two approaches. We will lose some of those comparisons, and we have decided to.

And the label does nothing for leads the crew never sees. Honest uncertainty about a lead is not coverage of the ones that never appeared.

How to tell if a vendor's AI is allowed to say it

You can check this in an evaluation without reading any code. Four questions do most of the work.

1. What labels can a finding carry, and who assigns them? If the answer is "the model rates its confidence", the model is grading itself.
2. Show me an unconfirmed finding in a sample report. If there is no such thing, the tool either never hesitates or hides its hesitation.
3. What happens to the wording of an unconfirmed finding? Look for impact language the evidence does not support.
4. Show me the stored evidence behind three findings I pick. Our post on penetration testing evidence covers what that evidence should contain.

A vendor that answers all four cleanly has built a system that can say "I don't know". A vendor that changes the subject has answered too.

The position

"Honest before impressive" is our tagline, but it started as a design constraint. An AI pentester that cannot say "I don't know" will say something else instead, and the client pays to find out it was wrong. We would rather lose the demo than win it with a finding we cannot show.

Frequently asked questions

Why do AI pentesting tools hallucinate vulnerabilities?

Language models are trained and scored in ways that reward guessing over admitting uncertainty. In a pentest loop, a model will often turn a real observation into a confident but wrong conclusion. An error message or a timing difference is enough. The fix is a check outside the model that decides what counts as evidence.

What does needs review mean in an Arxiis report?

It means the crew saw signs of an issue but the evidence did not bind to a record in the tamper-evident execution ledger. It is not a lower severity. It is a separate output that a human reviews before anyone acts on it.

Is a needs review finding a false positive?

Not necessarily. Some needs review findings are real issues that could not be confirmed in the target environment. The label says the evidence is not there yet, and a human decides whether to investigate, re-run or close it.

Why doesn't Arxiis give each finding a confidence score?

A confidence score is the model grading its own work, and thresholds on it can be tuned to change the finding count. Arxiis uses two labels instead. A finding is proven when its evidence binds to a ledger record. Otherwise it is needs review, which tells the reader a human must look before anyone acts.

Does labelling findings needs review lower Arxiis's benchmark score?

Yes. On XBOW-104 Arxiis solves 56% raw and 44% confirmed, and the confirmed figure is the one we lead with. The same rule gives a measured false-positive rate of 0% on XBOW-104 sweeps.