NPM Test And Verification Scripts

_The command-runbook layer of the helm-expt test map._

The npm scripts are the repository's proof harness. They answer three questions:

Are the committed artifacts internally consistent?
Do the installer packages still render what the receipts say they render?
Which catalog/data files need to be regenerated after a change?

Most scripts use Node.js built-ins and do not need npm install. Some scripts shell out to cub, cub installer, helm, kubectl, kind, or ConfigHub when they are exercising installer, upload, or live-cluster paths.

This page is the human runbook. The generated inventory of every current package.json script is:

• NPM Script Catalog
• NPM Script Catalog CSV

Regenerate it after adding, removing, or renaming npm scripts:

npm run npm-scripts:catalog
npm run npm-scripts:catalog:verify

Test Cadence

Use the narrowest check that proves the change you made. The full verifier is the release gate, not the default inner-loop command.

Live tests are not the default proof loop. Run them when the goal is to create or refresh live evidence:

Treat live runs as evidence-producing work. Commit the receipt and regenerate the summary that indexes it, or do not claim the lane changed.

Run the strict live parity lane serially. Its cleanup step owns kind clusters for that lane, so concurrent parity runs can remove another in-flight run's cluster and produce misleading failures.

Some live rows need an explicit target profile. The profile is a property of the test cluster, not a hidden chart change.

Run --preflight first when using a guarded profile:

npm run live-parity:top20 -- --preflight --chart ingress-nginx --base default --target-profile kind-loadbalancer

kind-loadbalancer is guarded and refuses to run while another kind cluster is present, because cloud-provider-kind observes kind clusters host-wide.

Chart Refresh Cadence

The catalog should keep the currently supported chart version and any candidate new version separate until promotion evidence exists.

Supported catalog movement should be explicit:

current supported version
-> latest candidate render/proof
-> gap and quirk review
-> live or lifecycle evidence where needed
-> catalog promotion decision

Script Naming Rules

Common Commands

Where Helm And ConfigHub Actually Run

The strict two-cluster parity harness is the preferred default for proving that regular Helm and cub installer reach the same outcome for one chart/base row. The existing ConfigHub/OCI live comparator remains the delivery proof. Use data/lane-test-matrix/summary.md for the exact pass/missing state across the corpus.

If a pinned chart version is available through OCI but no longer appears in the classic Helm repository index, pass an OCI repository override:

npm run kind-parity:run -- \
  --chart bitnami/nginx --version 24.0.2 --base http-clusterip \
  --repo-url oci://registry-1.docker.io/bitnamicharts

The strict two-cluster and ConfigHub/OCI live parity runners treat oci://... repo URLs as OCI chart repositories and omit Helm's --repo flag.

The generated corpus control surface for these lanes is data/lane-test-matrix/. Its doctrine is docs/reference/lane-test-doctrine.md.

For live reruns, use data/live-parity-rerun-plan/summary.md. It keeps infrastructure blocks, runtime watch rows, and possible parity defects separate before anyone changes a recipe.

User-Install Verification

These scripts are for an outside user who followed the Redis demo and wants to prove their own result matches the catalog.

They are not the repo-wide proof harness. npm run verify uses the committed per-chart proof scripts and generated-data verifiers. Redis participates in that same chain with redis:verify-proof, redis:verify-package, and redis:verify-confighub-proof; redis:verify-install:* is an extra user-demo check because Redis is the first human quick-start path with an install-checks.yaml. The generic verify-install:* scripts are compatibility aliases for charts that have their own install-checks.yaml; they are not a repo-wide chart verifier.

These currently ship for Redis only:

recipes/bitnami/redis/25.5.3/install-checks.yaml

Other charts have their chart-specific proof/package scripts, but not Redis-style user-install checks until their own install-checks.yaml files are added.

User Bulk-Ops Verification

This script is for a user who followed the NGINX bulk scan and patch tutorial. It talks to the current ConfigHub organization, so it is not part of the offline npm run verify chain.

Per-Chart Proof Scripts

The top-20 curated charts have chart-specific scripts:

<chart>:compare
<chart>:verify-proof
<chart>:verify-proof:self-test
<chart>:verify-package
<chart>:generate-proof
<chart>:generate-package

Use them when working on one curated chart. For example:

npm run prometheus:compare
npm run prometheus:verify-proof
npm run prometheus:verify-package

Implementation note:

Redis remains bespoke because it carries the first user-install verification
path and Redis-specific quick-start checks.

The other 19 top-20 chart scripts share scripts/lib/proof-kit.mjs. Their
per-chart files are declarative specs plus chart-specific assertions.

That means a proof-model change usually belongs in the shared proof kit, while a chart-specific object assertion, value choice, or scan finding belongs in the individual <chart>-proof.mjs spec.

For non-curated charts, the broader generators and verifiers own the proof surface:

npm run next80:verify
npm run next80:verify:self-test
npm run verify:artifact-chain

The top-100 data keeps proof state separate from remaining capability work:

catalog_status       whether the chart has a maintained proof surface
not_yet_enabled      which recommended capabilities still have hard gaps

Catalog And Data Scripts

These keep the generated catalog/data surfaces current.

For the current regeneration order, see data/README.md.

Live And Runtime Scripts

These require a cluster or ConfigHub state. They are not part of the simple fresh-clone check unless they are run in verify-only mode against committed receipts.

Standalone runtime tests live beside this file:

tests/chart-install-test
tests/chart-install-sweep
tests/existing-secret-proof

Those are for cub + cluster execution. The npm scripts are the repo verification and artifact-generation layer.

Recommended Checks By Change Type

Current Top-100 Reading

The top-100 is not a single readiness state.

100 charts have recipe/package proof artifacts.
20 charts are catalog-supported for the declared local-test scope.
80 charts are proof-grade, but not yet catalog-supported.
54 charts are variant-rich.
46 charts are default-only.
25 charts have at least one hard gap for a recommended extra capability.
37 charts have buildable variant backlog where the path is known but not run.

Use data/top100-catalog-analysis/summary.md, data/chart-facts/summary.md, and data/model-completeness/summary.md together:

top100 summary       = what proof surface exists
chart facts          = what quirks and hard gaps each chart has
model completeness   = whether the chart has Level-2 support under declared scope
variant backlog      = which useful variants still need build work