SLSA Level 3 implementation¶

When Ludmilla presented the supply chain security roadmap to the engineering leadership, she put a single requirement on the board: every artefact Golem Trust ships must carry verifiable proof of how it was built, signed by something that cannot be forged by a compromised build worker. SLSA Level 3 is the framework that defines exactly what that proof must contain and how robust it must be. This runbook describes what Level 3 requires, how Tekton Chains satisfies each requirement, how to verify SLSA provenance for a given image, and how to diagnose common failures.

SLSA framework overview¶

SLSA (Supply-chain Levels for Software Artefacts) defines four levels of progressively stronger supply chain guarantees. Golem Trust targets Level 3 for all production container images. The levels build on each other:

Level 1: build is scripted, provenance exists
Level 2: build service generates provenance, provenance is authenticated
Level 3: build service is hardened, provenance is non-falsifiable
Level 4: hermetic builds, two-person reviewed build instructions (not currently required)

Level 3 requirements and how Tekton meets them¶

Scripted build: all Tekton Pipelines are defined as YAML resources in the golem-trust/platform Git repository. No manual build steps exist. The pipeline definition is the authoritative build script, reviewed and version-controlled.

Build service: Tekton Pipelines running on the dedicated build cluster is the designated build service. Developers do not build production images on their workstations. The build cluster is the only entity with push credentials to the Harbor golem-trust/ project.

Isolated builds: each TaskRun executes in a freshly created Kubernetes Pod. When the TaskRun completes, the Pod is destroyed. There is no persistent build agent that could accumulate state across builds or be compromised over time. Ludmilla describes this as “every build starts from a clean sheet, like a golem with a fresh chem.”

Provenance available: Tekton Chains generates a provenance attestation for every TaskRun and stores it as an OCI artefact in Harbor alongside the image.

Provenance authenticated: the attestation is signed by Tekton Chains using a key that is stored in Kubernetes as a Secret, accessible only to the Chains controller ServiceAccount. The signing key is generated by Cosign and its public key is published at https://golemtrust.am/.well-known/cosign.pub.

Provenance non-falsifiable: the provenance is generated by Tekton Chains running inside the build cluster, not by the build Task itself. A compromised build Task cannot alter the provenance because it has no access to the Chains signing key. The provenance records the exact inputs to the build (source commit SHA, base image digest) and the outputs (image digest), all of which are cryptographically bound.

Attestation format¶

Tekton Chains produces SLSA provenance in the slsa/v1 format. A representative provenance document for a Golem Trust image looks like:

{
  "_type": "https://in-toto.io/Statement/v0.1",
  "predicateType": "https://slsa.dev/provenance/v0.2",
  "subject": [
    {
      "name": "harbor.golems.internal/golem-trust/example-service",
      "digest": {
        "sha256": "a1b2c3d4e5f6..."
      }
    }
  ],
  "predicate": {
    "builder": {
      "id": "https://tekton.golems.internal/chains/v2"
    },
    "buildType": "https://tekton.dev/attestations/chains/pipelinerun@v2",
    "invocation": {
      "configSource": {
        "uri": "git+https://gitlab.golems.internal/golem-trust/example-service",
        "digest": {
          "sha1": "abc123..."
        },
        "entryPoint": "tekton/pipeline.yaml"
      },
      "parameters": {
        "git-url": "https://gitlab.golems.internal/golem-trust/example-service",
        "git-revision": "main"
      }
    },
    "buildConfig": {},
    "metadata": {
      "buildStartedOn": "2026-03-20T09:00:00Z",
      "buildFinishedOn": "2026-03-20T09:04:32Z",
      "completeness": {
        "parameters": true,
        "environment": true,
        "materials": true
      }
    },
    "materials": [
      {
        "uri": "git+https://gitlab.golems.internal/golem-trust/example-service",
        "digest": {
          "sha1": "abc123..."
        }
      }
    ]
  }
}

Verifying SLSA provenance with slsa-verifier¶

Install slsa-verifier from the Golem Trust internal mirror:

go install github.com/slsa-framework/slsa-verifier/v2/cli/slsa-verifier@v2.6.0

Retrieve the image digest for the image under verification:

IMAGE_DIGEST=$(crane digest harbor.golems.internal/golem-trust/example-service:latest)

Run the verifier, specifying the builder identity and the source repository:

slsa-verifier verify-image \
  harbor.golems.internal/golem-trust/example-service@${IMAGE_DIGEST} \
  --source-uri git+https://gitlab.golems.internal/golem-trust/example-service \
  --builder-id https://tekton.golems.internal/chains/v2 \
  --print-provenance

A successful verification prints PASSED: Verified SLSA provenance followed by the provenance document. Any failure prints the specific condition that was not met.

Common Level 3 failures and diagnosis¶

Attestation missing from Harbor: the image was pushed but no .att tag exists alongside it. Check whether the Chains controller is running and whether it has permissions to push to Harbor. The service account tekton-chains-sa must have push rights to the golem-trust/ Harbor project.

kubectl logs -n tekton-chains deployment/tekton-chains-controller --tail=200 | grep -i error

Signature verification fails with key mismatch: the image was signed with a key that does not match the published public key. This can happen if the Chains signing key was rotated without updating the published key. Retrieve the current public key from the Chains Secret:

kubectl get secret signing-secrets -n tekton-chains -o jsonpath='{.data.cosign\.pub}' | base64 -d

Compare this against the content at https://golemtrust.am/.well-known/cosign.pub. If they differ, the published key must be updated via the standard change request process, with Ludmilla’s sign-off.

Builder ID mismatch: the provenance records a builder ID that does not match the expected value. This typically means the Chains configuration builder-id field was changed without updating the verification policy. Check the current Chains configuration:

kubectl get configmap chains-config -n tekton-chains -o yaml

Source URI mismatch: the provenance records a different source URI than the one passed to slsa-verifier. This can happen if the pipeline was triggered by a mirror or fork rather than the canonical repository. Angua treats any source URI mismatch on a production image as a potential supply chain incident until proven otherwise.

Missing materials: the provenance materials list is empty or incomplete. This indicates the TaskRun did not record its inputs correctly. Check that the source-checkout Task is setting the CHAINS-GIT_COMMIT and CHAINS-GIT_URL results, which Tekton Chains reads to populate materials.

Provenance for TaskRun in wrong namespace: Chains only processes TaskRuns in namespaces listed in its watched namespaces configuration. Production builds must run in the tekton-pipelines namespace.

Kyverno admission check¶

Kyverno enforces SLSA compliance at admission time. No image may be deployed to the production namespace unless slsa-verifier confirms Level 3 provenance. The Kyverno policy lives in golem-trust/platform/kyverno/slsa-policy.yaml. If a deployment is rejected with a Kyverno admission error referencing SLSA, the most common cause is an image that was pushed manually outside the Tekton pipeline. Sam Vimes Jr. from the security team should be notified of any such incident.