The Checkbox Trap
Executive Order 14028 told every federal supplier to produce Software Bills of Materials. Most teams responded the way enterprises always respond to mandates: they found a tool, bolted it onto CI, and declared victory.
That is not a strategy. That is a checkbox.
A checkbox SBOM is generated once, attached to a release artifact, and never looked at again. It satisfies the letter of the requirement while ignoring the spirit entirely. When an auditor asks "show me the provenance of every dependency in your Q3 release," a checkbox SBOM cannot answer.
What Auditors Actually Want
CISA's SBOM guidance and the NIST Secure Software Development Framework (SSDF) are converging on a clear standard. Auditors are not asking "do you have an SBOM?" They are asking:
- Completeness — Does your SBOM include transitive dependencies, build tools, and vendored code?
- Freshness — Is the SBOM generated at build time from the actual dependency resolution, or from a stale lockfile?
- Attestation — Is the SBOM cryptographically signed and tied to a specific build artifact?
- Queryability — Can you search your SBOM corpus to answer "which of our services use log4j 2.14?" in under five minutes?
If you cannot answer yes to all four, your SBOM strategy has gaps.
The Three Failure Modes
1. Generate-and-Forget
The SBOM is created during CI but stored as a build artifact that nobody indexes. When a CVE drops, the security team has to manually check each repository. This is the most common failure mode and it scales terribly.
2. Wrong Stage, Wrong Format
Many teams generate SBOMs from source manifests (package.json, go.sum) rather than from resolved, built artifacts. The result is an SBOM that describes what you intended to build, not what you actually shipped. These are two different things.
3. No Provenance Chain
The SBOM exists but is not cryptographically tied to the release artifact. An attacker who compromises your artifact registry can swap binaries without invalidating the SBOM. This is the supply chain attack that SBOMs are supposed to prevent.
What a Defensible Strategy Looks Like
A defensible SBOM strategy has four layers:
-
Build-time generation — SBOMs are produced by the build system from resolved dependencies, not from source manifests. This happens automatically on every release.
-
Cryptographic binding — Each SBOM is signed and its digest is embedded in the release manifest alongside the artifact hash. Tampering with either invalidates the release.
-
Centralized indexing — SBOMs are shipped to a searchable store (we use Elasticsearch) so that vulnerability queries across the entire portfolio resolve in seconds, not hours.
-
Policy gates — Releases that fail SBOM completeness checks or contain known-vulnerable dependencies are blocked before they reach production. No exceptions, no manual overrides.
This is exactly what ReleaseFlow was built to do. The immutable release manifest binds SBOMs, signatures, and policy attestations into a single, auditable unit that travels with the artifact from build to deployment.
Key Takeaways
- An SBOM that is not generated at build time from resolved dependencies is unreliable
- An SBOM that is not cryptographically signed is not tamper-proof
- An SBOM that is not indexed is not queryable during incident response
- A defensible SBOM strategy covers generation, signing, indexing, and policy enforcement
The supply chain threat landscape is not slowing down. Neither should your SBOM maturity.
Ready to move beyond checkbox compliance? Talk to us about building a defensible software supply chain.