1. Overview¶

Chainguard sells minimal, continuously rebuilt container images (now branded Chainguard Containers) whose goal is zero or near-zero known CVEs, plus signed provenance and SBOMs for every image. Everything is built from source on Chainguard's own Linux "undistro", Wolfi, using two open-source tools the company maintains: melange (builds packages) and apko (assembles images from packages).

The pitch to a platform team is simple: stop patching base images yourself. You swap FROM python:3.12 for FROM cgr.dev/chainguard/python, and the image you pull tomorrow already contains yesterday's upstream fixes. Your scanner output drops from hundreds of findings to a handful, and every image carries a Sigstore signature you can enforce at admission time.

The company has since grown well beyond images. As of late 2026 its portfolio includes Libraries (rebuilt-from-source Python, Java and JavaScript packages), VMs, Chainguard OS Packages, a unified Repository, hardened CI Actions, Agent Skills, and a migration assistant called Guardener (product blog). This guide focuses on the container story, which is where most teams start, and covers the rest briefly in section 7.

Who it suits: teams with compliance pressure (FedRAMP, PCI DSS 4.0, CMMC, SLSA), teams drowning in scanner noise, and anyone who wants a signed, minimal base without maintaining a distro. Who it suits less: workloads that depend on Debian/RHEL-specific packaging, or teams unwilling to pay once they need pinned version tags.

What you'll do in the lab (section 8): compare CVE counts, verify signatures and SBOMs, port a Rust service to a distroless multi-stage build, assemble an image with apko, package software with melange, and enforce signed Chainguard images in Kubernetes with Kyverno.

2. Why minimal, rebuilt images matter¶

Most CVEs in a typical container scan come from the base image, not your code. A stock debian- or ubuntu-based language image ships a shell, a package manager, coreutils, perl, and dozens of libraries your app never calls, and each one is a scanner finding and a potential foothold.

Three forces make this worse over time:

• Distro lag. Stable distros backport selected fixes on their own schedule. Many CVEs sit as "won't fix" or "no fix yet" for months, so your scanner stays red even after a fresh pull.
• Image drift. Teams pin a base image and forget it. Every week it sits, newly disclosed CVEs accumulate against packages already inside it.
• Attack surface. A shell and package manager in production let an attacker who gains code execution download tools and pivot. Distroless images remove that.

Chainguard attacks all three. Images contain only the runtime and its dependencies; packages track upstream releases closely rather than a frozen distro snapshot; and images are rebuilt continuously, so pulling by tag picks up fixes automatically.

Be precise about what "zero CVE" means. It means zero known, unfixed CVEs in the packages Chainguard ships, as reported by mainstream scanners, at build time. It says nothing about your application dependencies, your own code, or a CVE disclosed tomorrow. It is a strong baseline, not a guarantee, and Chainguard publishes a shared responsibility model for exactly this reason.

3. Wolfi: the "undistro"¶

Wolfi is a community Linux distribution designed only for containers: it has no kernel, no init system and no installer. It is the package layer underneath every Chainguard image, and it is fully open source at packages.wolfi.dev.

Two compatibility rules trip people up early. First, Wolfi packages cannot be mixed with Alpine packages: same apk command, different C library and repositories. Second, package names often differ from Debian or Red Hat names; Chainguard maintains a package name mapping reference.

You can use Wolfi directly, for free, without any Chainguard account: cgr.dev/chainguard/wolfi-base is a tiny image with a shell and apk, and the public repository works with apko and Dockerfiles. Chainguard's commercial Chainguard OS Packages layer adds the enterprise repository behind paid images.

4. Chainguard Containers¶

The catalog holds hundreds of images (language runtimes, web servers, databases, Kubernetes components, AI/ML stacks) served from the registry cgr.dev. Browse it without signing in at the Chainguard Directory, which flags free, FIPS and STIG-hardened variants per image.

Variants¶

The distroless production images are rebuilt nightly and have no shell or package manager; the -dev variants add both so you can install build dependencies. The standard pattern is a multi-stage build: compile in -dev, copy the artifact into the distroless image.

Useful base images when you bring your own binary: static (for fully static binaries; no libc), glibc-dynamic (glibc and friends for dynamically linked binaries), and wolfi-base (a minimal shell + apk image).

Tiers and access¶

The free tier only exposes :latest and :latest-dev; version tags like python:3.12 are a paid feature. Catalog Starter lets teams pick five non-FIPS images free to trial in real workloads, but cannot be combined with an existing license and excludes full support and the CVE SLA. Paid organizations pull from a private path, cgr.dev/<your-org>/<image>, authenticated with chainctl or a pull token.

Practical implication of :latest-only free images: your builds are not reproducible unless you pin by digest (@sha256:...). Pin digests in production manifests and let Renovate, Dependabot or Chainguard's Digestabot bump them.

5. Supply-chain guarantees¶

Every Chainguard image ships with cryptographic evidence of what it is and how it was built, stored as OCI artifacts next to the image in the registry. You verify it with standard Sigstore tooling, no Chainguard account needed for public images.

Keyless signing in one paragraph. Chainguard's CI job authenticates to Sigstore's Fulcio CA with its OIDC identity and receives a short-lived certificate binding that identity to a signing key. It signs the image digest and records the signature in Rekor, a public transparency log. When you verify, you don't check a long-lived key; you check that the certificate's identity (the workflow) and issuer (the OIDC provider) match what you expect, and that the Rekor entry exists. Getting those two strings right is the whole game, and the lab shows them.

SLSA and compliance. Chainguard documents its builds against SLSA 1.1 and publishes mappings for PCI DSS 4.0, CMMC 2.0 and FedRAMP, plus STIG-hardened and FIPS variants. Treat these as inputs to your own compliance evidence, not a substitute for it.

CVE SLA. Paid customers get a contractual remediation SLA for critical and high CVEs in covered images; the free tier and Catalog Starter do not. Check your contract for exact timelines.

Scanner quirks. Occasionally a scanner flags a CVE Chainguard has already fixed or marked not-affected, usually because the scanner's Wolfi data lags. Chainguard keeps a guide for that case; updating the scanner DB fixes most of them.

6. Tooling¶

Six tools cover nearly everything you will do with Chainguard. Four are open source and work with no account.

apko, conceptually¶

apko is declarative image assembly. You list repositories, a keyring, packages, users, entrypoint and target architectures; apko resolves and installs packages directly into a filesystem and writes the OCI layers. There is no shell execution during the build, so builds are fast, reproducible (same inputs give the same digest), and automatically produce an SBOM. The trade-off: anything that is not an apk package has to become one first, which is melange's job.

melange, conceptually¶

melange is the package build system. A melange.yaml declares package metadata, build-time dependencies, and a pipeline of steps (fetch a tarball and check its hash, run cargo build, make install, go build, etc.). It runs each build in an isolated sandbox and outputs signed .apk files plus an index. Wolfi itself is roughly a few thousand melange files in the wolfi-dev/os repository.

chainctl highlights¶

• chainctl auth login then chainctl auth configure-docker wires Docker credentials for cgr.dev.
• chainctl auth pull-token create makes a long-lived pull token for CI or a cluster pull secret.
• chainctl images list, chainctl images tags list, chainctl images history explore what your org can pull.
• chainctl images diff shows package and CVE changes between two image digests, useful for change review.
• chainctl starter init / add-images manage a Catalog Starter selection.

7. The wider product family¶

Beyond containers, Chainguard now applies the same build-from-source model to other layers of the stack.

Chainguard Libraries was free with no commitment until June 30, 2026; check current terms before you plan around it. For most platform teams the realistic adoption order is: free images → Catalog Starter or paid containers → Custom Assembly → Libraries.

8. Hands-on lab¶

The lab takes about 2–3 hours and uses only free images and open-source tools. Each exercise builds on the last; exercises 5–7 can be done independently if you are short on time. Commands assume Linux or macOS on x86_64; swap x86_64 for aarch64 on ARM.

8.0 Prerequisites¶

apko and melange run from their own Chainguard images, so you do not need to install them. Create a working directory:

mkdir -p ~/cg-lab && cd ~/cg-lab

8.1 Exercise 1: Measure the difference¶

Goal: see the CVE and size gap between a stock image and its Chainguard equivalent.

docker pull python:3.12
docker pull cgr.dev/chainguard/python:latest
docker pull cgr.dev/chainguard/python:latest-dev

docker images --format '{{.Repository}}:{{.Tag}}  {{.Size}}' | grep -E 'python'

grype python:3.12
grype cgr.dev/chainguard/python:latest
grype cgr.dev/chainguard/python:latest-dev

# Summarize by severity
for img in python:3.12 cgr.dev/chainguard/python:latest; do
  echo "== $img"
  grype "$img" -o json | jq -r '.matches[].vulnerability.severity' | sort | uniq -c
done

Record your results (numbers vary by day; that variation is itself the lesson):

Checkpoint: expect hundreds of findings on the Debian-based image and zero or near-zero on the Chainguard production image. The -dev variant will show slightly more than production because it carries a shell and apk.

8.2 Exercise 2: Explore a distroless image¶

Goal: understand what "no shell" means in practice, and what the image defaults are.

# This fails: there is no shell in the production image
docker run --rm --entrypoint sh cgr.dev/chainguard/python:latest -c 'echo hi'

# The -dev variant has one
docker run --rm --entrypoint sh cgr.dev/chainguard/python:latest-dev -c 'id; cat /etc/os-release; apk info | head'

# Inspect defaults: user, entrypoint, workdir
docker inspect cgr.dev/chainguard/python:latest \
  --format 'User={{.Config.User}} Entrypoint={{.Config.Entrypoint}} WorkingDir={{.Config.WorkingDir}}'

# Run python directly: the entrypoint is the interpreter
docker run --rm cgr.dev/chainguard/python:latest -c 'import sys, os; print(sys.version, os.getuid())'

Checkpoint: the production image runs as UID 65532 (nonroot), the entrypoint is the interpreter itself, and /etc/os-release in the dev image reports Wolfi.

Debugging tip: to poke at a running distroless container, attach an ephemeral debug container that shares its process namespace instead of adding a shell to the image: docker run --rm -it --pid=container:<name> --network=container:<name> cgr.dev/chainguard/wolfi-base sh. In Kubernetes, use kubectl debug -it <pod> --image=cgr.dev/chainguard/wolfi-base --target=<container>.

8.3 Exercise 3: Verify signatures, SBOMs and provenance¶

Goal: prove the image came from Chainguard's pipeline and read exactly what is inside.

IMG=cgr.dev/chainguard/nginx:latest

# 1. Verify the keyless signature (public catalog identity)
cosign verify \
  --certificate-oidc-issuer=https://token.actions.githubusercontent.com \
  --certificate-identity=https://github.com/chainguard-images/images/.github/workflows/release.yaml@refs/heads/main \
  "$IMG" | jq '.[0].optional | {Issuer, Subject}'

# 2. Verify and extract the SPDX SBOM for one architecture
cosign verify-attestation \
  --type https://spdx.dev/Document \
  --certificate-oidc-issuer=https://token.actions.githubusercontent.com \
  --certificate-identity=https://github.com/chainguard-images/images/.github/workflows/release.yaml@refs/heads/main \
  --platform linux/amd64 \
  "$IMG" | jq -r '.payload' | base64 -d | jq '.predicate.packages[] | {name, versionInfo}' | head -40

# 3. Look at build provenance
cosign download attestation --platform linux/amd64 \
  --predicate-type https://slsa.dev/provenance/v1 "$IMG" \
  | jq -r '.payload' | base64 -d | jq '.predicate.buildDefinition' | head -40

# 4. Negative test: a wrong identity must fail
cosign verify \
  --certificate-oidc-issuer=https://token.actions.githubusercontent.com \
  --certificate-identity=https://github.com/attacker/repo/.github/workflows/x.yaml@refs/heads/main \
  "$IMG" && echo 'UNEXPECTED PASS' || echo 'Correctly rejected'

Checkpoint: step 1 prints the GitHub Actions issuer and Chainguard's release workflow as subject; step 4 is rejected. If step 2 or 3 returns nothing, list what is attached with cosign tree "$IMG" and adjust the predicate type. Images pulled from a paid org path (cgr.dev/<org>/...) are signed by a different identity; use the issuer and identity from Chainguard's verification guide for those.

8.4 Exercise 4: Port a Rust service to a distroless multi-stage build¶

Goal: take a typical single-stage Dockerfile and convert it to a Chainguard build stage plus a distroless runtime. The service uses only the standard library, so it builds offline and stays tiny.

cargo new hello --bin 2>/dev/null || mkdir -p hello/src
cd hello

Cargo.toml:

[package]
name = "hello"
version = "0.1.0"
edition = "2021"

[profile.release]
lto = true
strip = true

src/main.rs:

use std::io::{Read, Write};
use std::net::TcpListener;

fn main() {
    let addr = std::env::var("BIND").unwrap_or_else(|_| "0.0.0.0:8080".into());
    let listener = TcpListener::bind(&addr).expect("bind failed");
    println!("listening on {addr}");
    for stream in listener.incoming() {
        let Ok(mut stream) = stream else { continue };
        let mut buf = [0u8; 1024];
        let _ = stream.read(&mut buf);
        let body = "hello from a distroless Chainguard image\n";
        let resp = format!(
            "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: {}\r\n\r\n{}",
            body.len(),
            body
        );
        let _ = stream.write_all(resp.as_bytes());
    }
}

The "before" Dockerfile most teams start with, Dockerfile.before:

FROM rust:1
WORKDIR /app
COPY . .
RUN cargo build --release
EXPOSE 8080
CMD ["./target/release/hello"]

The Chainguard version, Dockerfile:

# ---- build stage: -dev variant has a shell, cargo and a C toolchain ----
FROM cgr.dev/chainguard/rust:latest-dev AS build
# build stage only: root is needed to write /work; the runtime stage stays nonroot
USER root
WORKDIR /work
COPY Cargo.toml ./
COPY src ./src
RUN cargo build --release

# ---- runtime stage: distroless, glibc only, no shell ----
FROM cgr.dev/chainguard/glibc-dynamic:latest
COPY --from=build --chown=nonroot:nonroot /work/target/release/hello /usr/local/bin/hello
USER nonroot
EXPOSE 8080
ENTRYPOINT ["/usr/local/bin/hello"]

Build, run and compare:

docker build -f Dockerfile.before -t hello:before .
docker build -t hello:cg .

docker run -d --name hello-cg -p 8080:8080 hello:cg
curl -s localhost:8080

docker images --format '{{.Repository}}:{{.Tag}}  {{.Size}}' | grep hello
grype hello:before -o json | jq '.matches | length'
grype hello:cg -o json | jq '.matches | length'

docker rm -f hello-cg

Checkpoint: hello:before is typically over 1 GB with hundreds of findings; hello:cg is in the low tens of MB with near-zero findings. If cgr.dev/chainguard/rust is not in the free tier when you run this, keep the official rust:1 image as the build stage only; the runtime stage is what ships. For Go, the same pattern uses cgr.dev/chainguard/go:latest-dev and CGO_ENABLED=0 into cgr.dev/chainguard/static:latest.

Why it matters for production: pin both FROM lines by digest before merging (docker buildx imagetools inspect cgr.dev/chainguard/glibc-dynamic:latest prints it) and let a bot bump them.

8.5 Exercise 5: Assemble an image with apko¶

Goal: build an image declaratively from Wolfi packages, with no Dockerfile.

cd ~/cg-lab && mkdir -p apko-demo && cd apko-demo

apko.yaml:

contents:
  repositories:
    - https://packages.wolfi.dev/os
  keyring:
    - https://packages.wolfi.dev/os/wolfi-signing.rsa.pub
  packages:
    - wolfi-baselayout
    - ca-certificates-bundle
    - curl

accounts:
  groups:
    - groupname: nonroot
      gid: 65532
  users:
    - username: nonroot
      uid: 65532
      gid: 65532
  run-as: 65532

entrypoint:
  command: /usr/bin/curl

archs:
  - x86_64

docker run --rm -v "$PWD":/work -w /work cgr.dev/chainguard/apko \
  build apko.yaml curl-apko:dev curl-apko.tar

docker load < curl-apko.tar            # note the tag it prints, e.g. curl-apko:dev-amd64
docker run --rm curl-apko:dev-amd64 -sI https://example.com | head -1
ls *.spdx.json                          # apko wrote SBOMs alongside the tarball

Checkpoint: a working curl image, typically under 20 MB, with SBOMs you did not have to generate. Run the build twice and compare digests with docker inspect: identical inputs should produce identical image digests.

8.6 Exercise 6: Package your own software with melange¶

Goal: turn the Rust service from exercise 4 into a signed apk, then assemble it with apko. This is how Chainguard itself builds every image.

cd ~/cg-lab && mkdir -p melange-demo && cd melange-demo
cp -r ../hello ./hello

# Generate a package signing keypair (melange.rsa / melange.rsa.pub)
docker run --rm -v "$PWD":/work -w /work cgr.dev/chainguard/melange keygen

melange.yaml:

package:
  name: hello-rs
  version: 0.1.0
  epoch: 0
  description: Tiny HTTP hello service
  copyright:
    - license: Apache-2.0

environment:
  contents:
    repositories:
      - https://packages.wolfi.dev/os
    keyring:
      - https://packages.wolfi.dev/os/wolfi-signing.rsa.pub
    packages:
      - build-base
      - busybox
      - rust

pipeline:
  - runs: |
      cargo build --release
      install -Dm755 target/release/hello "${{targets.destdir}}/usr/bin/hello"

docker run --privileged --rm -v "$PWD":/work -w /work cgr.dev/chainguard/melange \
  build melange.yaml --arch x86_64 --signing-key melange.rsa --source-dir ./hello

find packages -name '*.apk'             # packages/x86_64/hello-rs-0.1.0-r0.apk

Now assemble it with apko using a local repository, apko-hello.yaml:

contents:
  repositories:
    - https://packages.wolfi.dev/os
    - '@local /work/packages'
  keyring:
    - https://packages.wolfi.dev/os/wolfi-signing.rsa.pub
    - melange.rsa.pub
  packages:
    - wolfi-baselayout
    - hello-rs@local

accounts:
  groups: [{ groupname: nonroot, gid: 65532 }]
  users: [{ username: nonroot, uid: 65532, gid: 65532 }]
  run-as: 65532

entrypoint:
  command: /usr/bin/hello

archs: [x86_64]

docker run --rm -v "$PWD":/work -w /work cgr.dev/chainguard/apko \
  build apko-hello.yaml hello-apko:dev hello-apko.tar
docker load < hello-apko.tar
docker run -d --name hello-apko -p 8081:8080 hello-apko:dev-amd64
curl -s localhost:8081 && docker rm -f hello-apko

Checkpoint: glibc was pulled in automatically because melange recorded the binary's shared-library dependencies in the package metadata. You never wrote a RUN line, and the SBOM lists hello-rs with your version. --privileged is needed because melange sandboxes builds; in CI, run it on a dedicated runner.

8.7 Exercise 7: Enforce signed Chainguard images in Kubernetes¶

Goal: make the cluster reject anything that is not a verified Chainguard image.

kind create cluster --name cg-lab
helm repo add kyverno https://kyverno.github.io/kyverno/
helm repo update
helm install kyverno kyverno/kyverno -n kyverno --create-namespace --wait

policy.yaml:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: chainguard-only
spec:
  validationFailureAction: Enforce
  webhookTimeoutSeconds: 30
  background: false
  rules:
    - name: allowed-registry
      match:
        any:
          - resources:
              kinds: [Pod]
              namespaces: [default]
      validate:
        message: "Images must come from cgr.dev/chainguard"
        pattern:
          spec:
            containers:
              - image: "cgr.dev/chainguard/*"
    - name: verify-chainguard-signature
      match:
        any:
          - resources:
              kinds: [Pod]
              namespaces: [default]
      verifyImages:
        - imageReferences:
            - "cgr.dev/chainguard/*"
          mutateDigest: true
          attestors:
            - entries:
                - keyless:
                    issuer: "https://token.actions.githubusercontent.com"
                    subject: "https://github.com/chainguard-images/images/.github/workflows/release.yaml@refs/heads/main"
                    rekor:
                      url: https://rekor.sigstore.dev

kubectl apply -f policy.yaml

# Allowed: signed Chainguard image (Kyverno also rewrites the tag to a digest)
kubectl run ok --image=cgr.dev/chainguard/nginx:latest
kubectl get pod ok -o jsonpath='{.spec.containers[0].image}'; echo

# Denied: wrong registry
kubectl run bad --image=nginx:latest

Checkpoint: the first pod is admitted with its image rewritten to cgr.dev/chainguard/nginx@sha256:...; the second is rejected with the policy message. Newer Kyverno releases prefer a per-rule failureAction field over validationFailureAction; both work at the time of writing. Chainguard also documents equivalents for OPA Gatekeeper and the Sigstore Policy Controller. In real clusters, scope the registry rule to also allow your own registry and sign your own images the same way.

8.8 Stretch goals¶

• [ ] Sign hello-apko yourself with cosign sign --key (or keyless in CI) after pushing to a local registry, then add a third Kyverno attestor for it.
• [ ] Generate a VEX statement with vexctl for a finding in hello:before and feed it to grype.
• [ ] Add a CI job that fails if grype --fail-on high finds anything in the runtime image.
• [ ] Configure Renovate to pin and bump the FROM digests in your Dockerfile.
• [ ] If you have an account: compare two digests with chainctl images diff.

8.9 Cleanup¶

kind delete cluster --name cg-lab
docker rmi hello:before hello:cg curl-apko:dev-amd64 hello-apko:dev-amd64 python:3.12 2>/dev/null
rm -rf ~/cg-lab

9. Migration guide, gotchas and troubleshooting¶

Most migrations fail on the same handful of assumptions baked into Debian- or Alpine-based Dockerfiles. Work through this checklist per image.

1. Find the matching image. Search the Directory, or use Chainguard's Image Matcher / Dockerfile Converter (or Guardener for bulk work).
2. Split into build and runtime stages. Anything needing apk add, compilers or a shell goes in a -dev stage.
3. Translate package names. apt-get install libpq-dev becomes apk add postgresql-dev or similar; check the mapping reference rather than guessing.
4. Fix the user. Images run as UID 65532. Files you COPY need --chown=nonroot:nonroot, and the app must not bind ports below 1024 or write outside its home or a mounted volume.
5. Check the entrypoint. Many images set the binary itself as ENTRYPOINT (for example python, nginx), so a CMD ["python", "app.py"] becomes CMD ["app.py"] or an explicit ENTRYPOINT override.
6. Pin by digest, automate bumps. Free images are :latest only; production manifests should reference digests updated by Renovate, Dependabot or Digestabot.
7. Re-test health checks. HEALTHCHECK CMD curl ... breaks in a distroless image with no curl; use an HTTP probe from the orchestrator or a tiny static health binary.

Common errors¶

Operational practices¶

• Pull through a mirror (Harbor, Artifactory, ECR, Nexus all have documented pull-through setups) so builds don't depend on cgr.dev uptime and you control retention.
• Keep -dev images out of production namespaces; enforce with the registry rule from exercise 7 extended to deny *-dev tags.
• For CI auth, prefer assumable identities (GitHub Actions, GitLab CI, AWS, Azure and Kubernetes OIDC are all supported) over long-lived pull tokens.
• Watch Chainguard's EOL grace period policy: when an upstream version goes end-of-life, its tag stops receiving rebuilds, and CVEs start accumulating again.

10. Comparison with alternatives¶

Chainguard's edge is speed of patching plus signed provenance on a glibc base; its cost is a paid tier for version tags and SLAs. The hardened-image market has become crowded, and free tiers differ sharply in what they assume about you. The table is a qualitative summary; verify current terms before deciding.

The last row matters more than it looks. Because Wolfi, apko and melange are open source, you can get most of the technical benefits for free if you are willing to run your own rebuild pipeline, sign your own images and track advisories yourself. What the subscription buys is that pipeline as a service: version-tagged images for hundreds of projects, FIPS builds, a contractual CVE SLA and support.

11. References¶

• Chainguard Catalog Starter
• Quickstart for Chainguard Containers
• Using the Chainguard Directory
• Verifying Chainguard images with cosign
• Shared responsibility model
• Package name mappings
• Scanner flags a fixed CVE
• Chainguard product blog
• wolfi-dev/os on GitHub and Wolfi packages
• Hardened image alternatives overview (CyberPanel)

Commands, policy YAML and the Rust example were written for this guide and have not been run against today's images. Image names, free-tier membership and signing identities change, so treat the checkpoints as what to expect, not guarantees.