# 009 binding-distillation — RESULT (harvested 2026-08-01, tier T3-exploratory) **Verdict: BLOCKED-BY-GATE. The candidate teacher (Qwen2.5-3B-Instruct) pooled mid-layer sentence representations do NOT bind thematic roles — max agent_patient cross-construction + lexical-holdout primary CI-lo = 0.510 across 20 cells (5 mid-layers × {mean, attention-received} pooling), far below the 0.6 gate. No transferable role code exists to distill from, so Phase 2 (distillation training) was NOT run. This is itself an informative negative (MANIFEST 051/059): a 3B decoder-LM's pooled reps join the C1 program's 26 sentence embedders + BART in failing the same brutal cell — pooling preserves a transferable SURFACE-POSITION signal but destroys role.** ## What ran - Phase 1 teacher-verification gate only (mandatory). Teacher = **Qwen/Qwen2.5-3B-Instruct** (36 layers, H=2048; strongest cached general LM ≤3B). One forward pass with hidden_states + attentions; pooled over CONTENT tokens (specials + pure-punctuation dropped) two ways: **mean-pool** and **attention-received-weighted**. Mid-layers swept at depth fractions {0.35,0.45,0.55,0.65,0.75} → L{13,16,20,23,27}. Battery's OWN probe functions (`binding_battery.cross_grid` / `within_cell`, imported as a library — 008 fast_probe pattern; identical Z ⇒ identical AUC) on the 2000 frozen `agent_patient` stimuli (+ 1200 `genitive` as a probe-power positive control). Probe seeds {0,1,2} enveloped, n_boot 200, linear readout. Wall 180 s on GPU0. Phase 2 NOT run (gate failed). ## Teacher AUCs (linear, seed-enveloped; the gate cell is `primary_lo`) ### agent_patient (gate task) | cell | within (ceiling) | primary auc | primary_lo | primary_hi | flip_lo | surface-X | |---|---|---|---|---|---|---| | **L13_mean** (best ceiling) | **0.756** | 0.522 | **0.510** | 0.535 | 0.505 | 0.748 | | L13_attn | 0.633 | 0.516 | 0.498 | 0.535 | 0.495 | 0.629 | | L16_mean | 0.735 | 0.499 | 0.486 | 0.510 | 0.472 | 0.695 | | L16_attn | 0.644 | 0.522 | 0.505 | 0.539 | 0.491 | 0.653 | | L20_mean | 0.746 | 0.515 | 0.500 | 0.527 | 0.478 | 0.701 | | L20_attn | 0.631 | 0.506 | 0.491 | 0.519 | 0.458 | 0.604 | | L23_mean | 0.724 | 0.513 | 0.495 | 0.528 | 0.487 | 0.681 | | L23_attn | 0.628 | 0.484 | 0.460 | 0.505 | 0.410 | 0.603 | | L27_mean | 0.716 | 0.503 | 0.484 | 0.521 | 0.465 | 0.672 | | L27_attn | 0.621 | 0.505 | 0.484 | 0.523 | 0.465 | 0.626 | **Max primary CI-lo over ALL 20 cells (the lenient gate statistic) = 0.510 (L13_mean) < 0.6 ⇒ FAIL.** ### genitive (positive control — SONAR binds this at ~1.0) - within-construction ceiling HIGH everywhere (0.67–0.82, best L16_mean 0.817) but primary at chance (all CI-lo ≤ 0.486; auc 0.486–0.517). flip_lo 0.40–0.49. surface-X 0.60–0.69. - The LM's pooled rep does not even carry transferable POSSESSOR binding — the failure is not agent_patient-specific; it is a general pooled-sentence-rep transfer failure. ## Interpretation (scoped) 1. **Probe has power, the null is about TRANSFER.** Within-construction role AUC is high (agent_patient 0.62–0.76, genitive 0.67–0.82) — a linear probe recovers "who is agent / possessor" comfortably *inside a single construction*. So the flat cross-construction cell is NOT an absence of role information nor a dead probe; it is failure to *generalize across constructions with held-out vocabulary*. 2. **The teacher is a surface-position reader — it fails exactly like the students.** Surface cross-construction transfers well (surface-X 0.60–0.75, top L13_mean 0.748) while the role primary sits at chance AND flipped-parity primary is at/below chance (flip_lo 0.41–0.51). That below-chance-on-flipped-parity signature is the diagnostic of a pooled representation that encodes *surface order* (a bag-of-positions) rather than a construction-invariant role variable — the identical pathology found in ladder organisms 001–008 and in C1's embedder/BART sweep. The would-be teacher has the same disease as the pupils. 3. **Mean-pool > attention-received on the ceiling** (mean retains ~0.1 AUC more within-role signal at every layer) but neither pooling nor any mid-layer transfers. Earliest swept layer (L13) carries the most poolable role signal; deeper layers erode it (approach to the unembedding). 4. **Feeds MANIFEST 051 (embedder-binding-sweep) / 059 (binding-vs-scale).** One clean data point: a 3B instruction-tuned *decoder* LM's mean-/attn-pooled mid-layer reps do NOT bind, consistent with the hypothesis that no-binding-under-pooling is universal across architectures and objectives — the culprit is the mean/weighted POOL (which is permutation-symmetric and so can retain a position-count signal but not an ordered role assignment), not the encoder family or scale. ## Consequence for the experiment 009's premise — *inherit a transferable role code by distilling from a teacher whose pooled rep verifiably binds* — is unfulfillable with this teacher: there is nothing binding to copy. Distilling z toward a rep that itself only encodes surface position would, at best, transfer surface position (which ladder organisms already have and which is NOT what the battery primary measures). Correctly STOPPED without training, per PROTOCOL's BLOCKED-is-a-valid-outcome rule. ## Brier - **(a)** P(some teacher rep passes the gate) = **0.15**; outcome FAIL (0) → Brier (0.15−0)² = **0.0225** (well-calibrated: low mass on the event that didn't happen; the reasoning that a generic LM's pooled rep can't beat purpose-built embedders on this exact cell held). - **(b)** P(distilled primary CI-lo>0.6 | gate passed) = 0.40 → **unscored** (conditional gate not triggered; correctly did not train). - **(c)** P(distilled val_f1≥0.60 | trained) = 0.60 → **unscored** (no training). - Mean Brier over the one scored prediction = **0.0225**. ## Limitations / what did NOT run - Phase 2 distillation training did not run (gate). No organism was trained; no 5e8-token run, no distilled battery numbers. - One primary teacher (Qwen2.5-3B-Instruct); fallback models (gemma-2-2b-it, Llama-3.2-3B-Instruct, Qwen3-1.7B) were NOT swept — the instruction was to pick the best ≤3B, and given the *positive control (genitive) also fails to transfer under pooling*, model-shopping for a pass would be fishing against a mechanism (pooling) that is model-agnostic. Logged as a deliberate stop, not a coverage gap. - POOLED reps only. This says nothing about whether the LM binds roles in its UNPOOLED per-token / last-token / cross-attention states — 006 showed role abstraction is *generation-side* and probe- invisible, so the binding may well live in the LM's token states and be destroyed only by pooling. - Post-hoc cell selection (best of 20) inflates the effective false-positive rate of the gate; it did not matter here (every cell failed), and the ceiling-based primary rule is reported alongside. - Rung A / T3-exploratory. ## Follow-up worth funding? **Y (redirected).** Not "distill from a pooled teacher" (dead — nothing binds under the pool). The live questions this surfaces: (1) does ANY LM state bind roles *before pooling* — probe last-token / per-content-token reps of Qwen2.5-3B on the battery (a mini-051-for-decoders); if yes, distill from the UNPOOLED signal (cross-attention or per-token) rather than a single pooled vector. (2) 059 GTR-T5 base→XXL scale sweep is now the clean way to ask if scale ever induces *pooled* binding — 009 contributes a strong 3B decoder negative. (3) 097 (pooling-symmetrization theory) predicts exactly this: a permutation-symmetric pool can carry a position histogram (surface) but not an ordered role map — 009 is empirical support; formalize the impossibility.