Continuous penetration testing should mean continuous authorization, not a scanner on a loop

Continuous penetration testing should mean two things at once. Tests run on a set cadence and on defined changes. And every run is authorized by a current written scope file and bound to its own evidence. Drop the second half and you have bought a scanner on a timer. This post sets out what we think the word should commit a vendor to, and how to write that into a contract.

"Continuous" is not a defined term

Three different services are sold under the same word. One is an automated scanner on a schedule. One is a retainer for human testers who take requests through the year. One is an AI-driven tool that runs repeatedly against the same targets. All three can be useful. They are not the same purchase.

The word commits nobody to anything. It does not say how often a test runs, what starts one or who authorized it. Nor does it say what a finding must carry before it reaches your engineers. A contract that says "continuous" and stops there will be met by whatever the vendor already delivers.

So the useful question is not "is it continuous?" Ask instead: what starts a run, who authorized it, and what does each finding prove?

Continuous versus annual: what actually changes

An annual test examines one version of an application on the dates it was tested. The report describes that version. Every release after it is untested until the next engagement, unless something else covers the gap.

A continuous program changes what the report describes. Instead of one snapshot, you get a sequence of runs, each tied to a version and a date. The value is in the sequence: what is new since the last run, what is still there, and what no longer reproduces.

Annual testing still has a job. A deep, human-led engagement is better at business logic and multi-step problems, which repeated automated runs handle worst. Most teams that move to a continuous program should keep a periodic deep test alongside it. We would.

Two triggers: the calendar and the change

A continuous program needs two kinds of trigger, both written down.

The calendar. A baseline cadence for each asset, set by how often it changes and how exposed it is. A public API that ships every week and an internal admin tool that changes twice a year should not share a cadence. There is no universal right frequency, and we would distrust anyone who offered one.

The change. Events that start a run outside the calendar. A useful starting list:

• A new externally reachable endpoint, route or API version.
• Any change to authentication, session handling or authorization logic.
• A new third-party integration that sends or accepts data.
• A new host or domain serving the application.
• A fix for a previously proven finding, which needs a retest.

Not every commit should trigger a run. Testing on every merge produces noise, cost and a review queue nobody clears. Tie triggers to changes in attack surface, not to activity.

Who declares that a trigger fired matters as much as the list. Name the owner, usually the engineering lead for the asset, and the notice period the tester needs.

Every run is still an authorized engagement

This is where "continuous" most often goes wrong. A program that began with a signed scope document a year ago is now testing hosts that document never named.

In Arxiis, the rule does not relax because a program is recurring. The crew will not touch a non-local target without a written SCOPE + ROE authorization file. The scope guard checks every outbound request against the current version of that file. A host added to the estate last week is out of scope until someone with authority adds it, however plausible it looks.

You can sign one file per run, or one standing file for the program period with its own windows and blackout dates. That is a governance choice. Either works. A standing file nobody updates does not. Scope changes should go through the same change-control steps as a one-off test. Record the new version before testing on the new host begins. Our rules of engagement template sets those steps out.

Human supervision stays too. When the crew wants to escalate, it raises an approval request and the run waits for a named operator. If no one answers, the run halts and reports rather than carrying on. So a continuous program with Arxiis is a series of supervised runs, not an unattended loop. We explain why in human-supervised isn't a compromise.

Each run also has a budget and time watchdog. When it fires, the run stops and writes a partial report that says it is partial. It never reports a finished test it did not finish. Over a year of runs that matters, so a contract should count a partial run as partial. Post 16 covers why stopping is the safe default, and the full set of controls is on our safety page.

Every run is still evidence-bound

Repetition creates a quiet risk: findings that are carried forward rather than observed again.

In Arxiis, each engagement's evidence is isolated, and each run writes its own records to the tamper-evident, hash-chained execution ledger. A finding is labelled proven only when its evidence binds to a ledger record. Otherwise it is labelled needs review. A finding does not inherit proven from last month's run. If it is still there, this run has to show it again.

That gives each finding three honest states between runs:

• New: not seen before, labelled by this run's evidence.
• Still present: proven again, with a fresh ledger record.
• Not reproduced: the earlier evidence no longer reproduces.

"Not reproduced" is not the same as "fixed". The fix may be real. The run may also have taken a different path, or hit its budget first. Close a finding on a retest that targets it and produces evidence, not on a missing result in a general run.

Store the final hash of each run's ledger somewhere your own team controls. The chain shows if a record was edited later. An auditor reviewing a year of runs will want to check exactly that.

Writing "continuous" into a contract

The definitions do the work. These are the clauses we would want to see, as buyer or as vendor:

1. Run. One engagement against named assets, under a current signed authorization file, producing its own report and evidence records.
2. Cadence. The baseline frequency for each named asset.
3. Change triggers. The list of events, who declares them, and the notice period.
4. Authorization. Who signs the scope file and how scope changes are approved. No run touches a host absent from the current version.
5. Supervision. The named operator, a response time for approval requests, and a backup.
6. Evidence. Every confirmed finding in every run carries its stored evidence. Proven and needs-review findings are reported separately.
7. Early stops. A run stopped by its budget or time limit is reported as partial and does not count as full coverage.
8. Retest. What closes a finding: a targeted retest with evidence.
9. Roll-up. A periodic summary of new, still present and not reproduced findings across runs.
10. Exclusions. What the program does not test, in writing.

Our AI pentesting RFP template has the evidence and control questions to ask before you reach this stage.

What continuous does not fix

Frequency does not add capability. If a tool is weak on a class of problem, running it weekly misses that class weekly.

Our own numbers make the point. On XBOW-104 (19 October 2026), Arxiis solves ~82% of the common web bugs, level-1 tier only. On hard chained and blind attacks it solves 0–42%, a known weak spot. A continuous program of Arxiis runs repeats that profile. It covers the common classes often. It does not become a different tool on the hard ones. That is the case for keeping a periodic human-led test.

Frequency does not widen scope either. Arxiis tests web applications and APIs inside the authorization file. Internal networks, social engineering and anything absent from the file stay untested, every run. An asset nobody listed is forgotten every month instead of once a year.

And frequency does not create remediation capacity. More runs produce findings sooner. If engineering cannot absorb them, the open list grows and the needs review queue goes stale. Size the cadence to the team that has to act on the results.

The test of a continuous program is not how often it runs. It is whether, a year in, you can point to the signed scope version and the evidence record behind every finding in the report.

Frequently asked questions

What is continuous penetration testing?

Continuous penetration testing is a program of repeated tests on a set cadence and on defined changes, rather than one test a year. We think it should also commit the vendor to two things. Every run is authorized by a current written scope file. Every finding is bound to that run's evidence.

Is continuous pentesting better than an annual pentest?

They do different jobs. A continuous program tracks change between releases and catches common problems sooner. A periodic deep human-led test is better at business logic and multi-step chains. Most teams benefit from both.

How often should a continuous penetration test run?

Set a baseline cadence per asset based on how often it changes and how exposed it is. Then add change triggers such as new endpoints, authentication changes, new integrations and fixes that need a retest. There is no single right frequency for every asset.

Does every continuous pentest run need its own authorization?

Every run needs a current, signed authorization. That can be one file per run or one standing file for the program period. Scope changes must be approved and recorded before testing begins. Arxiis will not touch a non-local target without that file.

Can Arxiis run continuous tests without anyone watching?

No. Each run is supervised. Escalation needs approval from a named operator, and if no one answers, the run halts and reports. A continuous program with Arxiis is a series of supervised runs, not an unattended loop.