engineering

The verdict belongs in the body, not the status code

Returning a policy failure as an HTTP error conflates two situations a pipeline must treat differently: the build being wrong, and the tool being unavailable.

TRUSTIVAN ·

There is an appealing shortcut in designing a security gate for CI: make the endpoint return a non-2xx status when the policy fails. A shell script can then be one line, because curl --fail already exits non-zero and the build stops on its own.

We considered it and rejected it, and the reason is worth writing down because the shortcut is common.

Two failures that look identical

A pipeline calling a security gate can be in one of several states. Two of them matter here:

  1. The build genuinely violates policy. There is a critical, fixable vulnerability in the image. The correct response is to stop and fix it.
  2. The gate could not answer. The service is down, the network is partitioned, the credential expired, a proxy returned a 502.

These call for opposite reactions. The first is a real signal that must not be overridden casually. The second is an infrastructure problem, and treating it as a security failure teaches a team to add || true to the step — which disables the gate permanently, including for case one.

If a policy failure is expressed as an HTTP error, a client cannot reliably distinguish them. Both are non-2xx. You can try to discriminate on the specific code, but now the meaning of your security gate depends on every proxy, load balancer and service mesh between you and it agreeing never to synthesise that code. They will not.

What we do instead

Requesting a verdict succeeds. The HTTP status describes the transport: did TRUSTIVAN receive your request, understand it, and record an answer. The answer itself is in the body.

{
  "result": "fail",
  "policy_version": 3,
  "explanation": [ … ]
}

The rule for a caller is then unambiguous:

  • Non-2xx — TRUSTIVAN did not answer. This is an availability problem. Retry it; escalate it; decide as an organisation whether an unavailable gate should block a release. That is a policy question about your infrastructure, and it is yours to make deliberately rather than by accident.
  • 2xx with result: "fail" — the build violates the policy. Stop.

A CI step ends up a few lines longer. Those lines are the entire point: they are where the distinction lives, and making them explicit means somebody had to decide what an outage should do.

The same reasoning, elsewhere

Once stated, this principle keeps applying.

A verdict is recorded, not recomputed. Asking twice returns the same answer, because two stages of one pipeline disagreeing about whether a build passed is a worse failure than either answer being wrong. The second request replays the first decision rather than re-running the evaluation against a backlog that may have shifted underneath it.

Bulk operations report per-item outcomes. Applying one action to four hundred findings is not usefully “successful” or “failed”. The response says what happened to each, because a caller that retried the whole batch on a partial failure would suppress three hundred and ninety-nine findings twice.

Enumeration states its guarantee. Listing findings by recency is exhaustive; listing by severity is not, because severity changes and rows move. Rather than documenting that and hoping, every response reports whether the traversal was stable, so a client can assert the property it depends on instead of assuming it.

The thread running through all of these is the same: when a caller is a program rather than a person, ambiguity does not get resolved by judgement. It gets resolved by whatever the program happened to do, silently, at three in the morning.