# RESULT — 051 embedder-binding-sweep (BEYOND-SONAR block opener)

**Tier: T3-exploratory.** GPU2/phys2. Harvested 2026-08-03. Verdict: **INSTRUMENT_FAILURE across all 6
embedders — NOT clean nulls.** The battery's positive-control (within-construction / genitive ceiling)
gate fails on every model, so the instrument cannot certify a "no-binding" claim. **The informative
twist: role information is WEAK in ALL tested small embedders even *within* construction (ceilings only
0.54–0.74), unlike SONAR** — these encoders barely represent thematic role at all.

## What actually ran
Ran the `stimuli_v2` binding battery UNCHANGED (only the encoder swapped) on 6 off-the-shelf sentence
embedders: `all-MiniLM-L6-v2` (22M, d=384), `all-mpnet-base-v2` (110M, d=768), `intfloat/e5-base-v2`
(prefix "query: ", d=768), `BAAI/bge-base-en-v1.5` (d=768), `thenlper/gte-base` (d=768),
`sentence-transformers/sentence-t5-base` (d=768). Tasks: agent_patient (headline) + genitive (positive
control). Seeds 0,1,2; readouts linear + mlp; n_boot 1000. `out/`: `BATTERY_RESULTS_051.md`, six
`binding_battery_*.json`, `run.log` (ends `[051] finished; 0 model(s) errored`), `DONE`.

**Note on the results MD**: ignore the FIRST section (MiniLM smoke, n_boot=200) — it shows a degenerate
grid with cross-constr 1.000 and is not a real result. All numbers below are the seeds 0,1,2 / n_boot=1000
runs.

## Headline numbers (agent_patient, best readout per cell)
Every model → **VERDICT: INSTRUMENT_FAILURE** (within-construction ceiling < 0.9 positive-control gate).

| model | within (ceiling) | primary (cross+lex holdout) | flipped-parity | surface-cross | genitive within |
|---|---|---|---|---|---|
| MiniLM     | 0.556 / 0.587 | 0.504 [0.49,0.52] | 0.507 | 0.518 | 0.606 / 0.657 |
| mpnet      | 0.544 / 0.577 | 0.508 [0.50,0.52] | 0.507 | 0.526 | 0.623 / 0.682 |
| e5-base    | 0.557 / 0.604 | 0.504 [0.48,0.53] | 0.507 | 0.536 | 0.649 / 0.735 |
| bge-base   | 0.553 / 0.588 | 0.507 [0.50,0.52] | 0.500 | 0.533 | 0.614 / 0.693 |
| gte-base   | 0.553 / 0.588 | **0.509** [0.50,0.52] | 0.504 | 0.528 | 0.629 / 0.702 |
| t5-base    | 0.553 / 0.590 | 0.497 [0.49,0.51] | 0.486 | 0.539 | 0.609 / 0.674 |

(within = linear / mlp; primary = cross-construction + lexical holdout, agent_patient.)

- **Within-construction ceilings 0.54–0.74** (mlp genitive tops out ~0.66–0.74) — all below the 0.9 power
  gate. The instrument therefore refuses to issue a null.
- **Primary (cross + lexical holdout) cells all ≈ 0.49–0.51** — dead chance, no order-invariant role code.
- **Surface-cross only weakly elevated (0.52–0.54)** — even the surface signal that transferred strongly
  in SONAR is barely above chance in these small embedders.
- **z_bag role = 0.500 everywhere** (stimuli sanity: label not lexically recoverable). ✓
- **flipped-parity ≈ 0.50** — no anti-transfer signature (which itself requires a surface reader that
  isn't clearly present here).

## Interpretation — "no binding found" vs "instrument cannot certify"
These are distinct, and the honest read is the second: because no model passes the positive-control power
gate, we **cannot** conclude "these embedders don't bind roles" — the probe may simply lack power on a
0.54-ceiling representation. What we CAN say is stronger and more surprising: **role information is weak
in all six small embedders even within a single construction**, so their pooled representations do not
cleanly encode thematic role at all. This contrasts with SONAR, where the within-construction ceiling is
high (role is present) and only the *cross-construction* binding is absent. So the sweep does not extend
the "surface-carries / role-absent" SONAR pattern to small embedders — it reveals a different regime where
even the surface/role signal is faint.

## Sanity gates / instrument controls
- **Probe power (positive control)**: FAILS for all 6 (within < 0.9, genitive cross+lex < 0.9) → each
  model correctly flagged INSTRUMENT_FAILURE rather than a null. The gate is doing its job.
- **z_bag**: role AUC = 0.500 (within 0.05 of chance) for every model — stimuli sanity holds.
- **0 models errored** (run.log).

## Brier vs frozen PREREG_LITE.md predictions
- **(a) P=0.90** — NO model clears primary CI-lo > 0.6 (no-binding universal) → **TRUE** (all primary CIs
  cap ≈ 0.52).
- **(b) P=0.85** — ALL *powered* models show surface-transfers / role-absent → **FALSE**. No model is
  powered (empty antecedent), and surface-cross is only ~0.52–0.54, not the predicted high surface signal.
  The prediction implicitly bet the models would be powered; they were not — this is the instrument-failure
  surprise, scored as a miss.
- **(c) P=0.70** — ≥1 model primary point > SONAR's 0.508 but < 0.6 → **TRUE (marginal)**: gte 0.509 >
  0.508 (within resampling noise; flagged as razor-thin).
- **(d) P=0.75** — instruction-tuned (e5/gte) no different from SBERT (MiniLM/mpnet) → **TRUE** (all
  primary ≈ 0.50–0.51, CIs fully overlap; binding absence is architectural, not objective-driven).
- **Brier(a–d) ≈ 0.22.** Cost concentrated in (b), which pre-registered powered models that never
  materialized — the informative failure at the heart of this row.

## Incident (honest disclosure)
The first pass was **killed mid-run by a `tmux kill-server` incident** (a duplicate 040 manager killed all
guest tmux sessions while debugging a libsndfile failure — the t43 breach, now a PROTOCOL hard rule). At
kill time the first pass had produced 5/6 batteries (through gte, model 5/6). The run was **relaunched
cleanly**; the second pass completed all 6 models with **0 errors**. (The results MD accumulates both
passes; the run.log is the clean second pass, 6 models.)

## Limitations
- Off-the-shelf default encoding (no row-053 role-retrieval prompts, which might restore power); one
  stimuli set; probe = linear + 1-hidden MLP.
- All six models are small (22M–110M); the ceiling failure may be capacity-linked. Larger / instruction
  embedders untested here.
- Because no model is powered, every "no binding" cell is uncertified — the result is an instrument
  characterization, not a binding null.

## Follow-up worth funding? **Y (narrow).**
The instrument-failure is itself the finding and it directs the next steps: (1) row 053 role-retrieval
prompts to try to lift the within-construction ceiling above 0.9 (restore power) before re-asking the
binding question; (2) larger sentence embedders to test whether the weak-role regime is capacity-bound;
(3) row 052 (LASER/LaBSE MT objective) as pre-registered. Until a small embedder passes the positive
control, "no binding in small embedders" stays uncertified.