What a full test run verifies - and what you can expect to be true

A repository document, rendered for the site. View source markdown.

Generated at: 2026-07-30T12:38:02.000Z UTC · source: committed helm-expt evidence for this rendered repository document.

This page is the acceptance contract of the catalog: when a chart passes a full test run, these are the properties you can rely on. Each property names how it is verified and at which layer, and the honest limitations are listed at the end. (Experimental & unofficial, like the rest of this repo.)

The verified properties

For a catalog chart that passes a full test run, you can expect:

  1. Helm-equivalent render. The chart's cub installer render is semantically equivalent to helm template - the same Kubernetes objects with the same key fields - apart from declared governance additions (a generated Namespace object; rendered Secrets are deliberately not stored in the artifact, see property 5). Verified by: Helm-equivalence receipts plus installer package setup checks. npm run redis:compare reruns the clearest fresh Helm-vs-installer example; most curated <chart>:compare commands verify installer setup against the stored Helm-rendered object set.
  1. Installs via cub installer on the maintained catalog path. For rows with the relevant receipts, ConfigHub stores the rendered YAML as reviewable records, publishes or applies it through OCI, and reconciles it through an in-cluster Argo or Flux controller into a running workload. Verified by: ConfigHub proof receipts, local-kind e2e receipts, runtime GitOps receipts, and the live parity lanes recorded in the outcome coverage.
  1. Deterministic, pinned, pull-based delivery. The OCI artifact is content-addressed (digest); the controller pulls and reconciles it, so drift self-heals. No plaintext secret ever sits in the artifact.
  1. Your chosen namespace is honored. cub installer setup --namespace X puts the workload in X (via the set-namespace transformer in the package). Limitation (F1b): complex charts with a namespace embedded in spec fields (RBAC subjects, webhook clientConfig) are not yet fully re-namespaced - install those in their canonical namespace for now.
  1. Secrets are never in the artifact. Default password-generating bases render a Secret that is not shipped in the OCI artifact (a security choice - better than helm install, which ships plaintext inline). Such charts therefore need the existing-secret / External-Secrets path to run under GitOps. Limitation (F3): a default password-generating base can reconcile while the workload cannot start until the secret is provided out-of-band or via External Secrets. Proven good path: redis reuse-existing-secret deploys fully healthy. The product path needs required-secret validation and typed secret references so this cannot become a silent green.
  1. Day-2 changes flow through the governed path. A change made through ConfigHub (e.g. replicas) republishes the OCI artifact and reconciles to the exact runtime field via Argo/Flux - no out-of-band kubectl edit. Verified by: live Day-2 proof (e.g. nginx replicas 1→2 → 2/2).
  1. Three-way agreement. ConfigHub revision ↔ controller (Argo/Flux) sync ↔ live cluster all match. A PASS requires exact runtime field proof, not just a controller "Synced".
  1. Every claim has a receipt. Render, equivalence, scan, upload, and live observation are recorded as committed proof artifacts you can inspect.

Outcome-First Reading

Read "full test run" as an outcome, not as a script name. For every supported chart default and declared main choice, the desired end state is:

New to cub? Install the cub CLI first. You can pull and render public catalog packages without an account. Commands that save or change ConfigHub data require you to sign in.

What this command does. cub installer is a released, open-source plugin for the cub CLI. cub installer setup pulls a catalog package and writes its Kubernetes files locally. It does not apply those files to a cluster; use kubectl, Argo CD, or Flux for delivery. The generated scripts stop before doing any work when the plugin or kustomize is missing.

fresh or receipt-backed Helm equivalence
-> deterministic cub installer setup
-> ConfigHub upload, scan, clone/check, and safe-ops receipts
-> live Kubernetes apply and observation
-> ConfigHub OCI through Argo or Flux with runtime evidence
-> live Helm-vs-ConfigHub comparison across Helm, ConfigHub/OCI, and ConfigHub/kubectl

Default-only verification is not enough when the catalog advertises another main choice. For example, an existing-secret, no-crds, server-only, ingress, HA, or storage-mode base needs the same outcome tracking as the default base. A derived ConfigHub variant needs clone/check/mutation receipts and live evidence for the post-render choices it claims to support.

What a full per-chart run exercises

render-equivalence → install (default base) → install (a non-default variant) → namespace honored → secret path → Day-2 edit → three-way agreement, each backed by a receipt. A chart "passes" only when every applicable row holds.

Current coverage (honest)

Known limitations / open items

Where to look