Skip to content

What Artifact Keeper does here, and what it does not

Artifact Keeper is the single place every artifact in this PoC comes from: the build host, the installer (Anaconda) and the edge node (a QEMU VM in this PoC) never talk to an upstream mirror or registry directly. That makes it the source of truth for content. It is not the thing that decides whether content is trusted: that is done by signatures created in the pipeline and checked by each client. This page draws that line. The API-level detail is in Artifact Keeper notes.

Stores

  • RPMs. Our signed edge-site-config packages in the hosted (local in the API) repository rpm-edge-site, with repodata generated by the server on upload.
  • bootc images. rocky-bootc-base and rocky-edge in the hosted OCI repository oci-bootc, path-based: host:30080/oci-bootc/rocky-edge:10.
  • cosign signatures. Stored as ordinary OCI manifests next to the image: the legacy sha256-<digest>.sig tags that podman, skopeo, bootc and the installer read, and, if cosign 3 is left at its defaults, Sigstore bundles reachable through the OCI referrers API. Artifact Keeper stores both correctly.
  • Public keys. The cosign public key, our RPM key and the Rancher and EPEL vendor keys in the hosted generic repository raw-edge-keys, readable anonymously.

Proxies

  • RPM proxy repositories for Rocky BaseOS, AppStream and extras, EPEL 10, RKE2 and k3s. Upstream RPM signatures are inside the packages, so they pass through untouched, and the vendor's repodata/repomd.xml.asc is passed through unchanged (byte for byte for Rocky and RKE2), so dnf can keep repo_gpgcheck=1 on proxied repositories. EPEL 10 and the k3s tree publish no repomd.xml.asc, so there is nothing to pass through.
  • OCI proxy repositories for quay.io (the Rocky builder image) and Docker Hub (every RKE2 system image and the demo workload, through a containerd mirror with a path rewrite).

Signs

One thing: hosted RPM repodata. Through the signing API, Artifact Keeper creates an OpenPGP key for rpm-edge-site that never leaves the server, signs repomd.xml, and serves repomd.xml.asc and the public repomd.xml.key. The signature carries a 7-day expiry and Artifact Keeper re-signs on request, so a mirror that copies repomd.xml.asc once will see it go stale. The RPMs themselves are signed in our build (rpmsign); Artifact Keeper's own package signing (sign_packages) was not used.

Does not sign or verify images

cosign in the pipeline signs each image digest right after it is pushed (signing/sign-image.sh). Artifact Keeper stores the .sig tag like any other manifest; nothing in its UI or API treats it as a signature, and it does not check one on push or pull. Its promotion rules with require_signature exist in the source but were not tried here.

Does not enforce anything

Enforcement is in the clients, each through its own policy.json (containers-policy.json) and dnf configuration:

Client Refuses
build host (podman, skopeo) an unsigned base image in FROM, an unsigned image in skopeo copy
installer (ostreecontainer) an unsigned or wrongly signed rocky-edge image
edge node (bootc upgrade) the same, before anything is staged
dnf, everywhere an unsigned RPM (gpgcheck=1) or unsigned metadata where it is signed (repo_gpgcheck=1)

A registry that served an unsigned image would be refused by every one of them; the same registry with the client policies removed would be trusted blindly. The policies are shown on Architecture.

Root of trust

The cosign key pair is generated on the build host by signing/gen-keys.sh; the private key never leaves signing/keys/. The public key reaches each verifier differently:

  • Build host: read from signing/keys/pub/edge-cosign.pub on disk.
  • Node: copied into the image at build time as /etc/pki/containers/edge-cosign.pub, so every later bootc upgrade uses the key that came with an image that was itself verified.
  • Installer: fetched by kickstart %pre with curl from http://10.0.2.2:30080/api/v1/repositories/raw-edge-keys/download/edge-cosign.pub.

The last one is trust on first use. The key arrives over plain HTTP from the same server that serves the image, so anyone who can change what that URL returns at install time (on the network path, or with write access to raw-edge-keys) can supply their own key and an image signed with it, and the installer would accept both. Everything after the install chains from that first key. To close the gap: pin the key's sha256 in the kickstart and check it in %pre, embed the key in the kickstart served by a trusted install server, or fetch it over TLS with a CA the installer already trusts.

What this PoC skips for production

  • Keys without passphrases. The cosign key has an empty password and the RPM key has no passphrase; both sit in plain files on the build host. Use passphrases, a KMS (--key awskms://..., hashivault://...) or an HSM.
  • No rotation procedure. Keys never expire. Rotation means a new key, re-signing (cosign allows several signatures per digest), shipping the new public key in an image signed with the old key, then dropping the old key from the policy.
  • No transparency log. Signatures are made with --tlog-upload=false and verified without Rekor, because an edge network has neither OIDC nor Rekor.
  • Plain HTTP. Signatures protect content, not tag-to-digest resolution, the unsigned EPEL and k3s metadata, or the key download in %pre.
  • Floating tag instead of digest pinning. Nodes install and upgrade from rocky-edge:10. An install should pin a digest (--url=...@sha256:...) and track the tag afterwards.
  • Docker Hub fallback. RKE2's generated hosts.toml keeps registry-1.docker.io as the fallback behind the Artifact Keeper mirror, so an air-gapped site must also block egress.

Adapting this to your environment covers how to close these.