Abdolmadjid Masoomi

What a Security Audit Does Not Cover

The scope is the whole content of the finding, and it is the part left off the badge

Published
2026-09-12
Length
3 min read · 610 words
Status
supported not independently verified

An audit report says something precise about a defined system at a defined moment. The badge on the website says something vague about a company forever. How to read the difference, and the four questions that recover the real meaning from a claim of having been audited.

What an audit actually is

A defined examination, of a defined thing, over a defined period, by a defined party, producing findings.

Every one of those definitions is a constraint, and the constraints are what make the result worth anything. An examination without a stated boundary cannot be checked, repeated, or disagreed with — which is a description of an advertisement rather than an assessment.

What the badge drops

The badge keeps one fact: an examination happened. It discards the scope, the date, the method and the findings.

That is precisely inverted. What it discards is the content; what it keeps is the part that is true of everybody who paid. And it converts an event into an apparent state — this company is secure, rather than this component was examined this way in March.

The four questions

What was in scope, and what was explicitly excluded? Exclusions are the more informative half and are rarely volunteered.

When, and what has shipped since? The finding describes the system on a date. Everything merged afterwards is outside it.

What kind of examination was it? A documentation review establishes that policies exist. A configuration check finds misconfiguration. An active attempt to break in finds what an attacker of that budget finds in that time. A source review finds implementation defects. These differ so much that treating them as one word does most of the damage.

Were findings published, including the fixed ones? A report showing only passes has been curated, and the curation is the finding.

Why scope is usually the answer

An examination of a login flow is not an examination of a product. An examination of production does not cover the pipeline that builds production. A review against staging tells you about staging, which may differ from production in exactly the way that matters.

A narrow scope is entirely legitimate — most good assessments are narrow, because depth and breadth trade against each other at a fixed price. A narrow scope presented as a broad one is not, and the presentation is done by the marketing page rather than by the assessors.

Time, and why it decays fast

The report describes a system on a date. An organisation deploying continuously may not have run that exact system for long afterwards.

So an assessment two years old is a statement about a system that no longer exists. Not cynicism — the ordinary arithmetic of shipping. The question is not whether the finding was true, but whether the thing it was true of is still there.

What a good disclosure looks like

Scope named, with exclusions. Date given. Method stated. Findings summarised with severity. What was fixed, and when.

Organisations do publish this. It costs them something — it invites questions, and it puts failures in writing — which is exactly why it is worth more than a badge. Nobody publishes their own findings by accident.

What audits genuinely catch, and what they structurally cannot

Catch: configuration errors, missing controls, known vulnerable dependencies, and whole classes of implementation mistake. These are real risks and finding them is real value.

Cannot: establish that a design is sound, predict how an operator will behave next year, or determine that nobody trustworthy has already been compromised. Those are not gaps in diligence. They are outside what an examination of a system at a moment can establish about people over time.

Close

Read the scope first. Everything else in the claim is downstream of it, and a claim with no scope is a claim about nothing in particular — which is why it was left off the badge.