UNOFFICIAL/EXPERIMENTAL - hand-reviewed audit snapshot, 2026-06-10.
Some tables mix "the source scan saw this quirk" with "the catalog models it" and "a receipt proves it." Those are three different numbers. This audit lists all three, per quirk, so undercounting and overclaiming are visible: top100-source-vs-modeled.csv.
Method
- source_top100_count - charts with the quirk among the 100 highest-ranked rows of the committed source feature scan (
data/top500-catalog-analysis/source/source-feature-scan.raw.json). Static analysis of pinned chart archives. - modeled_catalog_count - charts whose quirk is recorded in the maintained corpus (
data/chart-facts/chart-facts.csvcolumns,data/top100-readiness/readiness.csvsource-feature tokens, or the maintained hook queue). "Not tracked" means no maintained axis exists. - proof_observation_count - receipts or observations that exercise the quirk specifically. "Covered by render parity, not isolated" means the quirk's output is inside a proven render but no quirk-level claim exists.
No existing generator was modified; this is an independent snapshot. Numbers change when the sources change; re-derive before quoting.
Where We Risk Undercounting
- Five quirk axes are not tracked at all in the modeled corpus: APIService aggregation (5 source charts),
semverCompare(71),.Files.Get(31), non-exact dependency constraints (26), and remote dependencies (47). The last two matter most: dependency ranges and remote subcharts are refresh-survival and provenance risks with no modeled axis. - The two "top-100s" are different lists. Source rank and modeled proof rank disagree on membership (the modeled corpus has 37 CRD charts; the source top-100 has 24). Any table comparing the columns without saying which list it counts is wrong by construction.
- Top-20 rows carry no source-feature tokens, so modeled counts for lookup, capabilities, tpl, RBAC, and storage are systematically understated for exactly the charts with the most proof.
- Hook reality is wider than the queue. 11 source-top-100 charts carry hooks; 5 are modeled in the maintained queue, and 5/5 maintained queue receipts are now observed. The reviewed delta is in
data/hook-lifecycle-review/. Hook delete policies and weights are only tracked once a chart enters the queue - and the queue counts 4 weight-bearing charts where the source scan counts 3, a counting inconsistency to reconcile.
Where We Risk Overclaiming
- "Covered by render parity" is not quirk proof. tpl (88 charts),
semverCompare(71), required/fail (71), and.Files.Get(31) are inside proven renders for modeled charts, under one values profile and one pinned capability profile. Reading that as "tpl is proven" or "works on other Kubernetes versions" is an overclaim; 81 source charts touch.Capabilities. - Webhook and CRD runtime lifecycle proof covers 2 charts (cert-manager, External Secrets). It demonstrates the observation pattern, not webhook support across the 21 source charts that ship webhooks.
- Definitional drift can read as coverage. chart-facts
extension_slots(82) is broader than the scan's raw/extra-manifest count (65); the scan's generated-function count (60) is broader than modeled generated facts (29) and generated Secrets (23). Comparing across definitions inflates apparent coverage.
Suggested Next Actions
- Add modeled axes (or an explicit "won't track" decision) for the five untracked quirks, starting with remote/non-exact dependencies.
- Populate source-feature tokens for the top-20 rows, or document that chart-facts is their only quirk source.
- Reconcile the hook-weight counting difference between the queue and scan.
- Label every cross-table count with which top-100 list it uses.