Run live on 2026-07-05, both arms on the same kind cluster (hx-bench-1). The task: install redis 25.5.3, change the replica count to 2 (chart default 3), upgrade the chart to 27.0.0, and keep the change. Then: what record exists of why replicas is 2, discoverable from the system alone?
This is not a rigged "bare helm fails". A competent engineer with plain helm can keep the change. The benchmark measures the safety margin and the record, not just the happy path.
Bare arm (helm + kubectl, no substrate)
The agent installed redis 25.5.3 and set replica.replicaCount=2 through the helm release values (a values-redis.yaml file, not a kubectl edit) - the right way. Then the upgrade to 27.0.0 was run three ways to find the margin:
| How the upgrade was run | Replicas after | Outcome |
|---|---|---|
helm upgrade --version 27.0.0 --reuse-values (careful) | 2 | kept - survives when the flag is remembered |
helm upgrade --version 27.0.0 --set auth.password=… (flag forgotten) | 3 | silently reverted - STATUS: deployed, no warning |
The ops requirement vanished on one forgotten flag, with no error. The record of why replicas was 2 exists only if the engineer hand-wrote a helm --description (optional; often blank) and kept the local values-redis.yaml. Lose the file or forget -f/--reuse-values on any future upgrade and the requirement reverts, silently.
Pilot arm (the substrate)
The replica edit is a recorded revision (Staging departure: 2 replicas, a local decision that must survive upstream refreshes). The chart upgrade to 27.0.0 arrived as an UpgradeUnit revision that merged with the local edit; the departure survived by the merge semantics, not by anyone remembering a flag. Applied live to the bench cluster: redis-replicas at 2 replicas on chart 27.0.0.
Crucially, the why is enforced, not optional. cub revision list shows the edit, its change description, and the upgrade arriving, forever, queryable from the org with no local file to keep.
Score
| Dimension | Bare (careful) | Bare (one slip) | Pilot |
|---|---|---|---|
| Correct end state | yes | no (reverted) | yes |
| Silent failures | 0 | 1 (unwarned revert) | 0 |
| Survives without remembering a flag | no | no | yes (merge) |
| Record of why, from the system alone | optional (helm desc) | none | enforced (revision + change-desc) |
| Reproducible without a local file | no | no | yes |
The honest conclusion
Both arms can produce the right end state. They differ in what happens when a human is ordinary rather than careful. Plain helm keeps the change only while every future upgrade remembers a flag and a file; one slip reverts an ops requirement with no warning. The substrate keeps the change by construction and records why. The Pilot advantage is not "helm cannot"; it is "helm will not warn you when you forget, and keeps no enforced record of intent."