# 091 canonicalization-tests — RESULT (self-harvested, 2026-08-08, tier T3-exploratory)

**BUILD/tooling row (block L). Deliverable: `canon_check.py` — a property-based label/slot
CANONICALIZATION checker that CERTIFIES a role/slot-labeled stimulus set's conventions before
a binding probe is trusted. Packages campaign instrument lesson #2 (focal-swap convention).
Pure-stdlib, deterministic, seconds inline (no tmux/GPU/numpy). ALL 8 validation gates PASS
across all 4 battery tasks; 11/11 self-tests pass. Reproduces + catches the 017 and 020 bugs.**

## What the checker does
Given a stimulus set in the `stimuli_v2` binding-battery format, `certify(items)` runs five
property checks, each returning PASS/FAIL **+ the specific violation**; overall PASS iff all pass:

- **(a) focal-swap consistency** — canonical focal deterministic (no A==B ties) + symmetric
  under swap (`focal(A,B)==focal(B,A)`, invariant across a swap-pair). Convention pluggable
  (default = alphabetically-first, the battery's `load_task` rule).
- **(b) label balance** — equal cluster sizes (the exact `_prop_idxmat` invariant), swap-mates
  present with order actually swapped, `y_role`/`y_surf` balanced, per-family parity well-defined.
  **← the 017 prop-imbalance bug.**
- **(c) slot-index alignment** — `A_surface_first` equals the ACTUAL token order of the A/B
  fillers in the sentence. **← the 020 BOS off-by-one bug.**
- **(d) lexical-holdout disjointness** — train/test filler vocab actually disjoint.
- **(e) no leakage** — swap-pair word-multisets identical (z_bag role-blind), no single token
  correlated with `y_role` (|phi|>0.20), sentence length uninformative.

## Validation — does it catch the 017/020 bugs? YES
Run on the CLEAN `stimuli_v2` battery (all 4 tasks) and on deliberately BROKEN copies
(`validate_091.py` → `out/results.json`). Per-property verdict matrix `[a b c d e]`:

| case | agent_patient | expected | result |
|------|---------------|----------|--------|
| CLEAN (alpha-first) | `PASS PASS PASS PASS PASS` | all PASS | ok (PASS anchor) |
| CLEAN (alpha-**last**, valid-diff) | `PASS PASS PASS PASS PASS` | all PASS | ok (no false alarm) |
| inject prop-imbalance **(017)** | `PASS FAIL PASS PASS PASS` | (b) FAIL only | ok |
| inject off-by-one **(020)** | `PASS PASS FAIL PASS PASS` | (c) FAIL only | ok |
| inject leakage marker | `PASS PASS PASS PASS FAIL` | (e) FAIL only | ok |
| inject non-disjoint holdout | `PASS PASS PASS FAIL PASS` | (d) FAIL only | ok |
| inject A==B ties | `FAIL PASS FAIL PASS PASS` | (a) FAIL (+c side-effect) | ok |
| non-symmetric focal_fn | `FAIL PASS PASS PASS PASS` | (a) FAIL isolated | ok |

Identical pattern on genitive/causal/temporal. **8/8 gates PASS**: G-clean, G-017, G-020,
G-leak, G-disjoint, G-focal-tie, G-focal-asym, G-noFP. Each injected bug flips ONLY its target
property (specificity), so the checker has a low false-positive rate — confirmed by the
alpha-last valid-but-different convention certifying all-PASS, and by every non-target property
staying PASS under each injection.

**017 reproduction:** dropping the `BA`-order item from k propositions yields unequal
proposition cluster sizes + missing swap-mates → (b) FAIL, naming the offending props (this is
the exact `_prop_idxmat` "unequal items per proposition" assertion the row-017 run hit).
**020 reproduction:** rolling `A_surface_first` by one within a family desynchronizes the
label's positional claim from the sentence token order → (c) FAIL, naming the mismatched items
(the off-by-one between label positions and sequence positions that killed row 020).

**Structural note (why clean passes e so cleanly):** the battery's swap-pair symmetry makes
every token co-occur equally with `y_role∈{0,1}` — measured max |phi| = 0.000 and class-conditional
length identical (mean 7.147 both) on the real agent_patient set — so any planted marker
(phi→1.0) is caught with an enormous margin.

## Prereg predictions — Brier
All 7 predictions confirmed TRUE:
- P1 (0.97) clean → CERTIFY PASS — **T**
- P2 (0.95) imbalance → (b) FAIL only — **T**
- P3 (0.95) off-by-one → (c) FAIL only — **T**
- P4 (0.96) leakage → (e) FAIL — **T**
- P5 (0.93) non-disjoint → (d) FAIL only — **T**
- P6 (0.90) alpha-last → all PASS (no false alarm) — **T**
- P7 (0.90) non-symmetric focal / ties → (a) FAIL — **T**

**Brier = 0.0046** (7/7 TRUE; well-calibrated — a BUILD row where the deliverable's behavior is
deterministic, so high-confidence predictions are appropriate).

## What did NOT run / limitations
- Leakage (e) is a STRUCTURAL check: swap-pair bag identity + single-token phi + length. It
  catches single-word tells and any break of the z_bag role-blindness guarantee, but does NOT
  test distributed/nonlinear multi-token leakage — that remains the job of the battery's runtime
  `z_bag` control + `probe_power_kit` (row 090). Documented, not a defect.
- The A==B tie injection also (correctly) trips (c) because identical fillers destroy slot
  identifiability; the isolated (a)-only anchor is the non-symmetric focal_fn case.
- Checker validated on the `stimuli_v2` battery format specifically; other labeled-stimulus
  formats would need a thin adapter to the item schema (documented in README).

## Deliverable
`091-canonicalization-tests/`: PREREG_LITE.md, README.md, RESULT.md,
`src/{canon_check.py, validate_091.py, test_canon_check.py}`, `out/results.json`.
Checker + self-tests + README **mirrored to `campaign100/shared/`**
(`canon_check.py`, `test_canon_check.py`, `README_canon_check.md`) — later rows call
`import canon_check; canon_check.certify(items)` before trusting any binding-probe result.
CPU-only, deterministic, ~seconds inline; no box/GPU/tmux used.

## Follow-up worth funding? Y (narrow)
Wire `canon_check.certify()` as a pre-flight gate into the binding battery itself
(`load_task` could assert it), and extend the item-schema adapter to the ladder/SONAR
retained-state probe rows so every future binding probe self-certifies its labels.