Large Machine Handover

Updated: 2026-06-08

This is the handover for a Codex instance with enough local resources to run cluster and GitOps tests. It starts from public main.

First Principle

Do not claim "checked", "verified", "live", "all charts", or "all quirks" unless the exact scope has committed evidence and a verifier.

Use these words precisely:

The repo has useful proof. It does not yet have complete live proof.

Current Honest State

Use data/outcome-coverage/summary.md as the chart-recipe-variant truth source.

The generated status dashboard is the front door for current counts:

data/status-dashboard/summary.md
data/outcome-coverage/summary.md
data/outcome-coverage/base-outcomes.csv

As of the current generated matrix, the durable row-level facts are:

chart-recipe-variant rows: 156
render parity rows: 156 pass
ConfigHub proof rows: partial
local live rows: partial
GitOps/OCI live rows: partial, with pass/watch/blocked/missing tracked by exact chart/base
live Helm-vs-ConfigHub rows: partial, with pass/watch/blocked/missing tracked by exact chart/base
complete core lane rows: partial
100 charts have recipe/package proof artifacts.
20 top-20 chart-level ConfigHub proof receipt sets exist.
20 top-20 local-kind receipt sets exist for selected supported scope.
10 derived ConfigHub variants have live intended-state receipts.

Do not maintain exact live lane counts in this handover. They change as receipts land. Use the generated files above for current pass, watch, blocked, and missing counts. Use data/live-parity-rerun-plan/summary.md to choose the next non-pass live row to rerun.

The apparent tension between "20 top-20 receipt sets" and "18 row-level ConfigHub lane passes" is expected: the lane matrix is exact by chart-recipe-variant row and includes non-default choices. Never collapse the row-level gap into chart-level success.

What Was Just Completed

PR #155 and PR #157 added:

• Redis ConfigHub proof refresh through the common proof lane.
• Live intended-state receipts for all 10 work orders in the first derived expansion wave:
  • Redis: prod-us-east, staging-eu-west
  • NGINX: prod-us-east, customer-acme-prod
  • Prometheus: prod-us-east, staging-eu-west
  • Grafana: prod-us-east, customer-acme-prod
  • Vault: regulated-prod-us-east, staging-us-east
• npm run derived-variants:verify now verifies all 10 receipts.
• Demo proof docs now distinguish ConfigHub Units from Kubernetes Units plus installer record.
• Issue #156 tracks Grafana placeholder drift observed during clone.

Important limitation: all 10 derived receipts intentionally stop before target binding and live apply. They prove ConfigHub intended state, not runtime.

The current target-bound derived variant receipts are:

runs/derived-variant-target-bound/nginx-prod-us-east/receipt.yaml
runs/derived-variant-target-bound/nginx-customer-acme-prod/receipt.yaml
runs/derived-variant-target-bound/metrics-server-prod-us-east/receipt.yaml
runs/derived-variant-target-bound/prometheus-server-only-prod-us-east/receipt.yaml
runs/derived-variant-target-bound/prometheus-server-only-staging-eu-west/receipt.yaml
npm run derived-variants:target-bound:verify

Those receipts prove clean uploaded NGINX http-clusterip, Metrics Server default, and Prometheus server-only-ephemeral bases were cloned with cub variant create --target. The cloned workload Units were applied to ConfigHub OCI targets, Argo CD reconciled the derived Spaces, and Kubernetes reported the workloads ready. Metrics Server also records APIService availability. Prometheus now has both production and staging target-bound derived receipts. The receipts also record a product detail: the derived Space label changes to the requested operating variant, while cloned Unit labels still preserve the source base label unless a post-clone mutation changes them.

The first target-bound blocked receipt is:

runs/derived-variant-target-bound/redis-staging-eu-west/receipt.yaml

It records that Redis staging cannot yet be honestly called target-bound live: the work order asks for a namespace change and Redis Secret delivery, but those are not yet represented by checked post-clone mutations or secret/fact bindings in the derived path.

Known Sharp Edge

Grafana generated-passwords derived variants initially cloned deployment-grafana-grafana with one duplicated 9094 port rewritten to:

name: confighubplaceholder
protocol: confighubplaceholder

Both Grafana derived variants were corrected by copying the reviewed source Unit data back into the derived Unit. The receipts record the corrective update.

Follow-up issue:

https://github.com/confighub/helm-expt/issues/156

Do not present Grafana as a clean clone until that behavior is fixed or explained by product.

Start Here On The Large Machine

Run these first:

New to cub? Install the cub CLI first. You can pull and render public catalog packages without an account. Commands that save or change ConfigHub data require you to sign in.

git status -sb
npm run docs:verify
npm run lane-tests:verify
npm run top20:verify-confighub-proof
npm run derived-variants:verify
npm run derived-variants:target-bound:verify
npm run derived-variants:target-bound:summary:verify
npm run completeness:verify
cub version
cub context get -o json
cub space list -o json
cub target list --space "*" -o json

These are read-only except for normal local generated-process temp files. Do not mutate ConfigHub or clusters until you state exactly what will change.

Live Testing Definition Of Done

For a chart-recipe-variant row to move toward fully proven, produce receipts for:

1. Fresh Helm install in a live cluster.
2. ConfigHub kubectl/apply delivery in a live cluster.
3. ConfigHub OCI plus Argo or Flux delivery in a live cluster.
4. Semantic object comparison across Helm and both ConfigHub deliveries.
5. Runtime checks that match the chart's actual workload and quirks.
6. Cleanup or retention receipt explaining what remains.
7. Lane-matrix update that marks only the rows actually covered as pass.

Runtime checks must cover relevant quirks, not just "kubectl apply returned 0":

CRDs
hooks and lifecycle jobs
admission webhooks
cluster-scoped RBAC
generated Secrets and separated Secrets
existing Secret target facts
PVCs and StatefulSets
Service, Ingress, TLS, and LoadBalancer behavior
lookup/tpl/template weirdness
mutable image tags and digest overrides
namespace and storage class prerequisites
day-2 upgrade and rollback for stateful or hook-heavy charts

Next Codex Work Packet

Use large-machine-roadmap.md as the active roadmap. The shortest useful path is:

1. Attach targets to one prod and one staging derived variant and produce target-bound live apply receipts.
2. Expand live Helm-vs-ConfigHub parity to the next useful top-20 row.
3. Keep the receipt schema and verifier as the gate before expanding to more charts.
4. Update the lane matrix after every exact row moves from missing to pass.

If any part fails, record a blocked receipt or issue with the exact command, environment, and observed failure. A failed or blocked lane is valuable evidence as long as it is not described as a pass.

Human Navigation Problem

The repo is hard for humans to navigate. That is now a first-class workstream, not cosmetic cleanup.

The large-machine Codex should improve:

• README.md: answer "what am I getting?" before proof mechanics.
• CATALOG.md and site pages: per-chart promise cards.
• docs/README.md: doc map by audience and task.
• docs/user/: only human-facing docs.
• docs/reference/: stable definitions, lane doctrine, command surfaces.
• docs/planning/: roadmap, handover, issue mirrors, execution queues.
• stale docs: move, archive, or mark as historical so they cannot mislead.

Do not delete docs casually. Move or mark them only after checking references with npm run docs:verify.

Verification Tools Explained

New users need to know what each verifier proves:

The next docs improvement should make this table visible from a user-facing "How verification works" page.

GitHub Sync Expectation

Work in small branches, commit evidence plus verifier changes together, then sync frequently:

git fetch origin
git rebase origin/main
git push -u origin <branch>
gh pr create ...
gh pr merge ...

After each merge, re-check open PRs:

gh pr list --state open --json number,title,headRefName,mergeStateStatus,url