Abdolmadjid Masoomi

You Cannot Verify the Code You Are Running

Open source proves what was published, not what was executed, and the gap between those is where trust actually sits

Published
2026-09-12
Length
4 min read · 729 words
Status
supported not independently verified

Published source is treated as evidence that a program does what it claims. It is evidence about a repository. Between that repository and the binary on your machine sit a compiler, a build machine, a distribution channel and an update mechanism, none of which the source describes. What closes parts of the gap, and what remains open.

What publishing source actually establishes

Two things, both real.

It establishes what was in a repository at a point in time — a fixed, citable, comparable artefact that can be argued about.

And it makes review possible by anyone who decides to do it, which is a genuine structural advantage over software nobody outside a company has ever seen.

Neither of those is a statement about the thing currently executing on your device.

The four gaps

The build. A compiler and a toolchain on a machine you did not see turned that source into a binary. The source describes the input to that process and nothing about the process.

The distribution. What you installed arrived from a server, through mirrors and caches and a package index. The repository was not involved in the delivery.

The update. Whatever you verified once is replaced, automatically, by something you did not. Verification has a shelf life measured in release cycles, and almost nobody renews it.

The server. For anything with a hosted component, the code that matters most is the code you are never given.

Why the server gap is the largest

Publishing a client tells you what your device does. It says nothing about what happens after the request leaves it.

The server can retain what it said it discards, log what it said it does not, and behave one way under observation and another way otherwise. None of that is visible from the client, however carefully the client is read.

This is not an argument against publishing clients, which is useful and which I do. It is a correction of what gets concluded from it — an open client is evidence about the boundary, not about the interior.

Reproducible builds, and what they buy

If the same source reliably produces a byte-identical binary, independent parties can rebuild and compare hashes. A claim about the build becomes something checkable by people with no relationship to the builder.

That closes more than it first appears. It addresses the build gap directly, and it addresses substitution in distribution too — provided somebody actually performs the comparison, which is the clause that carries the weight.

The cost is honest work. Timestamps, absolute paths, file ordering, locale, parallelism nondeterminism — each has to be eliminated and kept eliminated as the code changes. It is sustained engineering, not a flag you set.

And it says nothing about the server, which remains where it was.

Signing proves origin, not behaviour

A signature establishes that an artefact came from the holder of a particular key and has not been modified since.

That is worth having and it is frequently read as something else — as vetting, as approval, as a statement that the contents are safe.

Code signed by a compromised key verifies perfectly. So does code signed by someone who meant exactly what they built. The signature distinguishes those two cases not at all.

Who is actually doing the reviewing

Anyone can audit it describes a permission, not an activity.

Some widely used code is reviewed with great care. Some is assumed to have been reviewed by somebody else — and the assumption is held by everybody at once, which is precisely the condition under which nobody holds it.

That is not an argument for closed software, which has the same gap plus no permission. It is an argument against treating availability of source as though it were a completed audit.

What this leaves you with

Prefer what can be checked over what must be believed, and notice which one you are being offered.

Prefer reproducible builds where they exist, and prefer projects where somebody other than the author publishes a matching hash.

Prefer software whose important parts run locally, because local execution is the only place the server gap does not apply.

And treat published source as a precondition for trust rather than a substitute for it — the thing that makes verification possible, which is a different claim from the thing having been verified.

Close

Verifiability is a property of a pipeline. One verifiable link does not make the chain verifiable, and the strongest link is usually the one people quote.

The useful question about any of this is not is it open. It is which link am I being shown, and who has checked the others.