Substack mockup · not published
Brandon Gilchrist · Defense at Machine Speed

Scan Twice

Ask a security program how a vulnerability dies and you will usually get a workflow: a ticket is opened, a fix is merged, the ticket is closed. Ask how anyone knows the vulnerability is actually dead and the room gets quieter. The ticket closed. The scanner, run again weeks later, may or may not flag the same path. The fix was reviewed by a person who read the diff, not by an adversary who tried to break it.

At machine speed that gap is fatal, because the other side does not close tickets. The other side retests. It is worth noticing that the offensive harnesses already work this way: ARTEX, the system from Part 2, ships a dedicated retest agent whose entire job is to return to claimed findings and render a verdict, reproduced, fixed, or inconclusive. The attackers built verification into the loop because unverified findings waste their time. Defense, with far more at stake, mostly verifies nothing.

This part is the argument for the second scan, and for the discipline it imposes: a fix counts only when the exploit path fails on a re-check.

The principle is simple to state and expensive to fake. A patch changes code. Whether it closes the vulnerability is an empirical question, and the only honest way to answer it is to run the attack again: the same exploit path, against the patched system, in a replica or a range, and watch it fail. Anything less, a code review, a scanner going quiet, a developer's assurance, is a claim. The re-attack is the proof. Programs that adopt this standard discover an uncomfortable number early: a meaningful share of "fixed" vulnerabilities are not fixed. The patch addressed the reported instance and left the class. It fixed the replica of the bug the scanner described and missed the variant the harness finds on the second pass. It broke under a slightly different input. You cannot know your real fix rate until you measure it adversarially, and most programs have never measured it at all.

Once you accept the standard, the cadence follows. One scan is an event, and events decay: the estate changes daily, new code ships, new vulnerabilities are disclosed, and last quarter's clean report describes a system that no longer exists. The defense loop that matches the threat is continuous and closed: scan, verify the findings by attempting the exploit, fix with assistance, re-attack to prove the fix, and feed every failure back into the next scan's priorities. Each pass through the loop makes the next pass smarter, because the failures are data: which codebases produce fixes that do not survive, which vulnerability classes recur, which teams' patches hold.

Four numbers describe a program running this loop, and they are worth adopting verbatim:

Notice what is not on the list: findings counts, scan coverage percentages, tickets closed. Those are activity metrics. The four above are outcome metrics, and the difference between the two lists is the difference between a program that performs security and a program that does it.

There is an organizational consequence hiding in the retest discipline. Verification has to be independent of the fixing, or it degrades into self-grading. The team that wrote the patch cannot be the authority on whether the patch holds, for the same reason the offensive harness's retest agent is a separate agent from its workers. In a small program this is a role, not a department: someone, or some system, whose only job is to try to break the fix and whose verdict closes the ticket. The ticket closes on proof, or it does not close.

The attackers scan you continuously and retest their own work as a matter of architecture. Scanning twice is the minimum viable reply: once to find, once to prove. Everything in the next part, deception, cost imposition, the economics of the campaign, builds on a defense that knows, rather than believes, that its fixes hold.