# 096 antibind-toy-theory — RESULT

**Tier T3-explanatory.** Deliverables: THEORY.md (derivation), src/toy.py + src/capacity_sweep.py
(numerical check), this file (fit to 001/006 + Brier). CPU/local (numpy), no GPU, no box.

## What we built
A minimal mean-pooled autoencoder over (agent, patient) triples with V fillers (THEORY.md
§1). Pooling destroys order up to a **weak residual channel** `g = β(e_a−e_p)`; the decoder
must recover which filler is the agent. Two codes compete: **BAG-LOOKUP** (memorize role
per content-bag `m`; zero generalization to novel fillers) vs **ROLE** (read `g`
filler-generally; O(1), generalizes). A "canonical-role" world (agent = lower-index filler)
opens the memorization route, mirroring 006's lexical shortcut, so the world does not force
either code — the optimizer chooses.

## Derived emergence condition
Description-length / capacity crossover (THEORY.md §3): distinct content-bags `≈V²/2`
overtake the effective memorization budget `min(T,K)` (K = decoder capacity), so

> **V\* ≈ √(2·min(T,K)/γ(β))**.

Below V\*, memorizing the bag is the cheaper low-loss basin (predicts 001 + 006's near-zero
novel accuracy at N=30/117). Above V\*, only the g-reader lowers loss → abstraction emerges
(predicts 006's N=1000 jump). Sharp because it is a threshold in V² → **sigmoidal in log V**;
same-vocab readout stays high throughout (the lookup table is always intact for seen fillers).

## Numerical check — main sweep (β=0.10, d=24, H=128, T tokens fixed, 2 seeds, novel-filler agent acc)
| V | 8 | 16 | 32 | 64 | 128 | 256 | 512 |
|---|---|---|---|---|---|---|---|
| T=1000  | 0.56 | 0.67 | 0.69 | 0.78 | 0.85 | 0.87 | 0.93 |
| T=4000  | 0.51 | 0.61 | 0.67 | 0.83 | 0.89 | 0.98 | 1.00 |
| T=16000 | 0.50 | 0.59 | 0.59 | 0.82 | 0.92 | 0.99 | 1.00 |

- **Qualitative bag→abstract transition reproduced (P3 ✔):** novel-filler role accuracy rises
  monotonically ≈chance(0.5) → ≈1.0 as V grows, at every T. `in-vocab` (same-vocab) accuracy
  stayed **0.98–1.00 across the whole sweep** and `train` accuracy **1.00** everywhere —
  exactly 006's signature (same-vocab ≈0.98 flat while novel-filler jumps). ablate-g ≈ 0.50
  (chance) in every cell — **the emergent rule reads the residual order channel** (P5 ✔; ablating
  it destroys role, reproducing "role rides surface order", 014–020/073).
- **001 predicted (P4 ✔):** at low V the loss-minimizing code is BAG-LOOKUP — train/in-vocab
  perfect, novel ≈ chance, no transferable role code. This is 001's "shortcut satisfaction."
- **Scaling law — P2 FALSIFIED, and the deeper story is cleaner.** Empirical
  V\*(novel≥0.75) = **64 for T=1000, 4000 AND 16000** — V\* is **independent of the data
  budget T** (larger T *raises* the whole curve, does not shift the knee). So it is not the
  coverage-limited √T branch. The follow-up **capacity sweep** (T=4000, vary decoder width H)
  shows V\* is **also flat in H**: interpolated V\*(0.75) = **44.5 / 46.5 / 44.4 / 49.0 / 49.1**
  at **H = 32 / 64 / 128 / 256 / 512** (fit slope d log V\*/d log H = **0.035**, vs 0.50 for √K).
  So the crossover is **independent of both T and K** over the tested ranges.
  The **dominant knob is β — the signal-to-noise of the residual order channel.** A β
  smoke-sweep at T=4000 moved the knee sharply: V\*(0.75) ≈ **~128 at β=0.08 → ~40 at β=0.15**.
  Interpretation: as V grows, each content-bag captures a vanishing *fraction* (∝1/V²) of the
  gradient, so the per-bag memorization signal dilutes while the **always-consistent** order-channel
  gradient (set by β) does not — abstraction wins once V is large enough that the shared signal
  outcompetes the diluted lookup. Because both signals scale with T, the *ratio* — hence V\* — is
  T-independent; because the g-reader is a single low-rank direction, it is ~H-independent too.
  Corrected emergence condition: **V\* ≈ √(c(d)/γ(β))**, a threshold set by embedding geometry
  and order-channel SNR, not by data or capacity.

## Capacity sweep (T=4000, vary decoder width H) — tests whether V*∝√K
| H | 32 | 64 | 128 | 256 | 512 |
|---|---|---|---|---|---|
| V\*(interp, novel=0.75) | 44.5 | 46.5 | 44.4 | 49.0 | 49.1 |

**Flat in H → √K FALSIFIED too.** Over a 16× capacity range V\* barely moves (44→49); the
power-law fit gives **d log V\*/d log H = 0.035** (√K would be 0.50). Combined with the T-sweep
(V\* flat in data budget), this locates V\* control in the **order-channel SNR (β)**, not any
memorization budget. Full numbers in `out/capacity.json`.

## Fit to 006 (the campaign's number)
006's transition is bracketed strictly by (117, 1000) fillers (novel-filler agent acc
0.00→0.05→0.73). The toy reproduces the **shape** (flat-low → sharp rise → high, sigmoidal in
log V, with flat same-vocab) and **explains** the mechanism (bag-memorization vs residual-order
reading). Absolute V\* has one free constant (effective per-bag budget min(T,K)/γ) not derivable
without SONAR's internals — **calibrated, not predicted**: choosing the log-mid of 006's bracket,
V\*≈√(117·1000)≈342, is consistent with V\*∈[117,1000] **by construction of the bracket** (G1
gate: the derivation is consistent with 006 and predicts a knee at a few-hundred fillers — a
falsifiable follow-up for a finer 006 N-grid). The load-bearing, non-fitted claims — the
qualitative transition, the flat-same-vocab signature, the order-channel dependence, and the
V² capacity-threshold form — all reproduce.

## Gates
- G1 (V\* ∈ [117,1000] by calibration): **PASS** (consistent; one free constant).
- G2 (monotone bag→abstract, not a flat null): **PASS** (0.5→1.0 at every T).
- G3 (in-vocab high across V): **PASS** (0.98–1.00 throughout).
- Ablation (zero g → collapse): **PASS** (ablate-g ≈ 0.50 in every cell).

## Brier (prereg-lite predictions)
| # | prediction | p | outcome | Brier |
|---|---|---|---|---|
| P1 | derived V\* ∈ [117,1000] (calibrated) | 0.80 | T (consistent by calibration) | 0.04 |
| P2 | crossover located near √T (within ~2×) | 0.65 | **F** — V\* is T-independent (capacity-limited) | 0.42 |
| P3 | qualitative bag→abstract transition reproduced | 0.80 | T | 0.04 |
| P4 | model predicts 001's low-V failure (bag code) | 0.85 | T | 0.02 |
| P5 | emergent rule reads residual order channel g (ablation) | 0.60 | T | 0.16 |
| **mean** | | | | **0.136** |

The miss is P2: I pre-registered the *coverage* branch (√T). The toy shows V\* is
independent of **both** T and decoder capacity H — the derivation's `min(T,K)` budget is not
the binding constraint over these ranges. The binding knob is **γ(β)**, the residual
order-channel SNR (V\*≈√(c(d)/γ(β))): a term already present in §3 but not identified as
dominant. The *competition mechanism* (bag-memorization vs order-reading) and all its
qualitative signatures survive; only the scaling variable was mis-called.

## Honest limits (T3)
Compression/optimization argument, not a proof about SONAR. Canonical-role device stands in
for the lexical shortcut (a fully balanced world forces abstraction at all V — wrong idealization
of the campaign's controlled stimuli). Absolute V\* calibrated, not predicted. Toy readout is
2-way retrieval (floor 0.5), so it cannot reproduce 006's *below-chance* 0.00 at N=30 (a
generation-OOV effect) — only the qualitative failure. No translation term (an MT decoder only
sharpens order-recovery pressure — same direction). Single architecture; H,β,d not exhaustively
swept beyond the capacity sweep.

## Follow-up worth funding? **Yes, conditionally.**
Corrected, falsifiable prediction for the real system: 006's transition location is governed by
the **strength of the residual order/position signal that survives pooling** (β in the toy), and
is roughly **insensitive to both training-token count and bottleneck width** — the opposite of a
naive capacity story. Discriminating experiments: (i) a data sweep at fixed capacity should NOT
move 006's knee (toy: T-independent); (ii) a bottleneck-width sweep (ties to 067) should also NOT
move it much (toy: H-flat); (iii) manipulating the *surviving order signal* — e.g. via 097's
pooling-symmetrization axis or position-encoding strength — SHOULD move it. That last is the
sharp, cheap test and directly links 096↔097.