Website UX Test Runbook

_A UX test (website walkthrough) in the helm-expt test map._

This runbook helps another Codex, Claude, or teammate start the generated helm-expt website locally and walk the maintainer through a UX review. The review starts with the top-level pages, then follows guide and doc links one or two clicks deeper.

Use it when the goal is to test whether the public site is understandable to a fresh Helm user. Do not use the walkthrough to rewrite the site live. First record findings, then make a focused follow-up change.

1. Sync And Verify

cd $HOME/code/helm-expt
git fetch origin
git pull --ff-only
npm run site:verify

If the site verifier fails, stop and report the failing page. Do not start the UX walkthrough from stale generated HTML.

2. Start A Local Static Server

Prefer port 8766.

python3 -m http.server 8766 --bind 127.0.0.1 --directory $HOME/code/helm-expt/site

Open:

http://127.0.0.1:8766/

If the port is already in use, pick another port:

python3 -m http.server 8767 --bind 127.0.0.1 --directory $HOME/code/helm-expt/site

3. Test From Another Laptop

127.0.0.1 only works on the machine running the server.

The simplest option is to run the same commands on the other laptop.

If the server should run on this machine and be viewed from another laptop on the same network, bind to all interfaces:

python3 -m http.server 8766 --bind 0.0.0.0 --directory $HOME/code/helm-expt/site

Find this machine's LAN IP:

ipconfig getifaddr en0

Then open this from the other laptop, replacing the IP:

http://<lan-ip>:8766/

If that does not load, check macOS firewall settings, VPN state, and whether both laptops are on the same network.

4. Top-Level UX Walkthrough

Ask the same question at every step:

Would a Helm user know what to do next without understanding the whole proof system?

Step 1: Home

URL:

/

Question:

Can a Helm user understand the promise in 30 seconds?

Expected message:

Use Helm charts. Ship ConfigHub variants.

Look for:

• a clear first action;
• plain explanation of why visible Helm stages matter;
• no requirement to understand every receipt before continuing.

Step 2: Try Now

URL:

/try.html

Question:

Is the first useful command obvious?

Expected:

• Redis is the simple path;
• kube-prometheus-stack is the serious chart path;
• commands are copyable;
• the page says what should happen after each command.

Step 3: Apps

URL:

/journey.html

Question:

Does the page explain how chart bases, variants, custom apps, existing apps,
platforms, and stacks become one application model?

Look for:

• a clear distinction between chart choice, variant management, app composition, and operations;
• a read-only path for existing Argo, Flux, KRM, rendered-manifest, or live cluster inputs;
• links to deeper app guides that tell the user what to type and what to expect.

Step 4: Status Matrix

URL:

/matrix.html

Question:

Does the matrix feel like a product/status page rather than a proof dump?

Check whether these states are understandable:

• pass;
• watch;
• blocked;
• not yet run;
• not applicable;
• deferred accepted.

Look for:

• F1/F2/F3/F4 rows are understandable;
• chart links and source links are useful;
• the next action column tells a human what to do.

Step 5: Hard Questions

URL:

/hard-questions.html

Question:

Are skeptical Helm-user questions answered plainly?

Check these topics:

• hooks;
• CRDs and webhooks;
• Secrets;
• upgrades;
• custom values and overlays;
• false-green GitOps sync;
• target prerequisites;
• free versus managed use;
• what happens when a public chart breaks the model.

Step 6: Chart Pages

URL:

/charts/index.html

Test Redis first, then kube-prometheus-stack.

Questions:

Does each chart page say what to try?
Does it say what is proven?
Does it say where the limits are?

Expected:

• Redis teaches the simple happy path;
• kube-prometheus-stack shows serious chart complexity;
• chart pages link back to matrix, evidence, and proof status.

5. Deeper Guide Pass

After the top-level walkthrough, test the pages a real user reaches within one or two clicks from home. Do not stop at a guide link just because the homepage looks clear.

For each page, ask:

New to cub? Install the cub CLI first. You can pull and render public catalog packages without an account. Commands that save or change ConfigHub data require you to sign in.

What am I trying to do?
What should I type?
What should I expect to see?
Do I need local files, a cluster created with `cub cluster up`, a bring-your-own cluster, a GitOps controller, or a ConfigHub account?
Is this proved, watch, blocked, planned, or not applicable?
Where do I go next?

Minimum depth sample:

Record whether each deeper page is:

clear enough
clear but missing a command
clear but missing expected output
too much raw proof data
confusing or contradictory

6. Record Findings

Use this format:

P0: Blocks a fresh user from understanding or trying the site.
P1: Makes the site confusing, overclaims, or hides a key product boundary.
P2: Polish, wording, layout, link, or navigation improvement.

For each finding, record:

Page:
Step:
Problem:
What the maintainer expected:
Suggested fix:

7. If Fixes Are Requested

Do not edit generated HTML by hand.

Edit the generator or source docs:

scripts/generate-public-site.mjs
scripts/generate-master-catalog-matrix.mjs
README.md
docs/user/README.md
docs/user/*.md

Then regenerate and verify:

npm run site:generate
npm run master-matrix
npm run site:verify
npm run master-matrix:verify
npm run docs:verify
git diff --check

Commit only focused changes that answer the UX finding.

7. Stop The Server

If the server is running in the foreground, press Ctrl-C.

If it is running in another shell, find and stop it:

lsof -nP -iTCP:8766 -sTCP:LISTEN
kill <pid>