UNOFFICIAL/EXPERIMENTAL. This entry keeps NVIDIA AICR v0.20.0 beside the v0.14.0, v0.18.0, and v0.19.0 entries. It uses the same EKS, H100, Ubuntu, training, and Kubeflow choices, so you can see what the new release changes without replacing the earlier configurations.
Why use this entry?
The two public OCI artifacts have different jobs:
| Artifact | What it contains | Digest |
|---|---|---|
| Source package | The AICR-generated app-of-apps Helm chart. | sha256:2eedf6b36ea2cfab828245e38f72b7ed344915b6695c6a56d92a744d25992431 |
| Literal configuration | The 17 exact Argo CD Application objects, one YAML layer per object. | sha256:4779ed69b9858791379a93704bb92ac11ec3b72e987b34921e293dd695415be8 |
Run this repository check to pull both artifacts without registry credentials and compare every pulled blob with the retained copy:
npm run aicr-v0200:public-verifyThe public OCI receipt records the exact references, digests, and anonymous pulls.
Three kinds of variant
The word variant appears at three different points. They are related, but they are not interchangeable.
| Variant | Meaning here | Current v0.20.0 status |
|---|---|---|
| Source variant | NVIDIA's curated combination of service, accelerator, operating system, workload, and platform: h100-eks-ubuntu-training-kubeflow. | Selected and retained. |
| Base variant | The checked starting configuration: source and intent, 17 exact Applications, lifecycle plan, field policy, digests, and receipts. | Retained in the Catalog, published as OCI, and stored in the live ConfigHub Catalog organization. |
| Derived variant | A reviewed change made from the base for development, staging, production, a region, or a customer. | Development, staging, and production variants now contain one promoted Grafana Secret change. |
The source record keeps three counts separate: 103 overlay files in the retained source tree, 102 entries in the embedded catalog, and 45 resolved leaves. The five choices select one of those leaves. NVIDIA curates the built-in source variants. Other Catalog providers can add target-specific configurations. A snapshot diff alone cannot decide what a particular target ought to contain.
The selection is now an imported record rather than metadata inferred by the ConfigHub Workshop generator:
| Field | Retained value |
|---|---|
| Provider | NVIDIA, acting as the source-catalog curator |
| Source catalog | NVIDIA AICR built-in catalog v0.20.0 |
| Catalog content digest | sha256:676f2d59eacd79ae1b72e5cbe00216b577def1da412dbdabb032f317a62dc1d8 |
| Selected source variant | h100-eks-ubuntu-training-kubeflow |
| Selection dimensions | service=eks, accelerator=h100, os=ubuntu, intent=training, platform=kubeflow |
The generated base-variant record copies those fields from the source-catalog record. The ConfigHub upload receipt carries the same provider, catalog digest, selected variant, and dimensions beside the exact-object comparison. This keeps the provider's source choice separate from the ConfigHub base and the later development, staging, and production changes. Provider-linked evidence is not treated as ConfigHub delivery or runtime evidence.
What the source produces
AICR resolves the selected source variant into a recipe with 15 ordered components. It then generates an Argo CD app-of-apps Helm chart. Helm renders that wrapper chart into 17 exact Application objects.
The digest index binds the selected inputs, source package, rendered Applications, and OCI manifests under one platform digest. The generation receipt records the commands and source version. The source verification checks the release checksum, signed binary provenance, recipe-catalog signature, and SBOM attestation.
Rendering and lifecycle work
The wrapper is only a partial flattening boundary. The 17 Application objects are exact. Sixteen of them point to Helm charts or local chart sources that a controller processes later.
The Catalog now materializes those 16 sources separately. Each record binds the fetched chart archive or local chart tree, the retained values, and the rendered object set with SHA-256 digests. Together they produce 409 local Kubernetes objects. Eight components contain 36 CRDs. No Helm hook object appears in this selected render.
These are local renders. They show what each source produces with the selected values. They do not show that Argo CD or Flux applied the objects, that the CRDs became ready, or that a workload ran on H100 hardware.
The route intent records the lifecycle work that may be needed: ordering, CRDs, prerequisites, and AICR health checks. The destination-specific records then say who performs that work for Argo CD and Flux. Neither record says that the work ran.
The Flux bundle builds locally into 29 controller objects. Its NVSentinel HelmRelease uses CreateReplace for CRDs because the component record declares NVSentinel as their sole owner. The setting is not copied to components that share CRDs. A live Flux CRD upgrade still needs a real Git source and target.
The field-policy assessment separates source-controlled choices from fields that may become reviewed ConfigHub changes.
Stored, changed, and promoted in ConfigHub
The ConfigHub base stores the 17 Application objects and a README as separate Units. Development, staging, and production are derived from that base.
The development change removes the literal Grafana administrator password and uses the existing Secret monitoring/aicr-grafana-admin instead. A dry run named the affected kube-prometheus-stack Application without changing stored data. One ChangeOrder then moved that change through staging and production. No other Application changed.
Every environment requires approval before release. An attempted unapproved release was blocked. After the production configuration and README were approved, ConfigHub published a release OCI with this manifest digest:
sha256:2576c38241454d44998c4ad26615552e9b04ed745e06724f4213c1c39a8e47d1The release was pulled back by that digest. Its configuration matches the promoted production object set and its README matches the approved production README. Promotion moved a reviewed change between ConfigHub environments; release publication made the resulting OCI available for delivery.
- ConfigHub base receipt
- Required-approval check
- Development-to-production promotion receipt
- Approved release OCI receipt
- Complete source-to-release chain
This proves the ConfigHub records, change, promotion, approval, OCI publication, digest pull, and file comparison. It does not prove that Argo CD or Flux reconciled the release, that EKS accepted the lifecycle work, or that an H100 workload ran.
What changed from v0.19.0?
The computed comparison records four source-version changes and no ordering change:
- the wrapper moves from 0.19.0 to 0.20.0;
kubeflow-trainer-postandnodewright-customizationsmove to 0.20.0;- NVSentinel moves from v1.9.0 to v1.20.0.
NVSentinel's health check now checks two driver-labelled DaemonSets as well as the labeler Deployment and pods. Its overall timeout changes from five minutes to 90 seconds, so a stalled DaemonSet reports the useful failure sooner. The optional zero-desired cases remain excluded. This is a source comparison; the health check has not run on EKS in this entry.
Current status
| Stage | Result | Record |
|---|---|---|
| Select the AICR source variant | Complete. The five criteria resolve to one provider-curated variant. | Source record |
| Verify the upstream source | Complete. Checksums, binary provenance, recipe-catalog signature, and SBOM attestation pass. | Source verification |
| Generate the wrapper and exact Applications | Complete. The retained result contains 17 Applications. | Generation receipt |
| Publish source and literal configuration OCI | Complete. Both artifacts pull anonymously and match the retained bytes. | Public OCI receipt |
| Materialize the 16 nested sources | Complete locally. The records bind 16 source artifacts and values sets to 409 objects, including 36 CRDs. | Nested source inventory |
| Resolve lifecycle work for Argo CD and Flux | Plans recorded. Both remain blocked until a destination, controller, and required credentials are available. | Route resolution summary |
| Retain a ConfigHub base and create derived variants | Complete. The base, development, staging, and production Spaces retain exact configuration and their own README Units. | ConfigHub base receipt and promotion receipt |
| Check approval, promote, and publish a release OCI | Complete. One named change reached production; an unapproved release was blocked; the approved release pulled back by exact digest and matched. | Required-approval check and release OCI receipt |
| Deliver through Argo CD or Flux | Not run for v0.20.0. | Requires destination-specific receipts. |
| Run on EKS and H100 | Not run. No training or NIM request, observation, or rollback is claimed. | Tracked in issue #1581. |
The configuration work is now retained through an approved production release. The next step is destination evidence: reconcile the release through Argo CD on EKS, check its CRD and prerequisite handling, run an H100 workload, record live state, and test exact rollback. Flux and small-fleet evidence follow that run.