# E03: sentence-broadcast decomposition of L24n token states — RESULT

Full run `full_v1` took 126 s on GPU 2 (2026-09-11). It used E01's 8,000 sentences and split, content tokens and
variance-weighted R², with 1,000-document bootstrap CIs. All sanity checks passed:
- the type-mean lookup reproduces .4553;
- s_{−t} excludes t;
- random-init F4 reproduces .93767.

**Adversarial check:** no separate breaker pass. The design has a built-in positive control: the same code path on
random-init states detects a .348 broadcast gain. That rules out the obvious failure for a negative result, a
misaligned or empty s_{−t}. The shuffled control (B3) is exactly 0.

## 4-line debrief
1. **Question:** is the half of SONAR's final token states that no local model explains a sentence-level broadcast,
   h_t ≈ f(w_t) + G s_{−t}?
2. **Answer:** no.
   - Adding the rest of the sentence's own final states (linear, full rank) raises content-token R² only from .474
     to .509 (+.035).
   - A learned rank-64 lexicon × sentence interaction adds +.016.
   - About .47 remains.
   - By contrast, the **random-init encoder is almost exactly own-token + broadcast**: .624 → .972.
3. **So:** training *removes* the broadcast structure. The trained final layer carries token-specific contextual
   information that is neither own-token nor a shared sentence state. Mean-pooling therefore averages ~.5 of
   variance in genuinely token-specific context features.
   - The simple fixed-point decomposition from IDEAS I6, z ≈ (I − G)⁻¹ mean f(w), is ruled out.
   - z is only .32 linear in the mean nonlinear lexicon.
4. **Next (per the registered decision rule):** find where the token-specific residual arises, layer by layer, and
   through which attention pathway. That links to E02's finding that the order code emerges between L20 and L22.

## Registered predictions

| id | p | registered | value [CI] | outcome | Brier |
|---|---|---|---|---|---|
| Q1 | .60 | R²(B1) − R²(B0) ≥ .20 | .035 [.034, .035] | FALSE | .3600 |
| Q2 | .85 | R²(B3 shuffled) − R²(B0) ≤ .02 | .0000 | TRUE | .0225 |
| Q3 | .65 | R²(B1) − R²(B4 lexical bag) ≥ .10 | .028 | FALSE | .4225 |
| Q4 | .35 | Z-lex pooled centred R² ≥ .40 | .322 [.318, .325] | FALSE | .1225 |
| Q5 | .50 | rank-64 G captures ≥ 90% of the B1 − B0 gain | .394 (r16 .22, r256 .72, r512 .93) | FALSE | .2500 |
| Q6 | .70 | random-init B1 − B0 ≤ .05 | .348 | FALSE | .4900 |

**Mean Brier .278, worse than chance (.25).**
- Q6 is the instructive miss. I expected a random transformer to have *less* sentence-level structure. In fact, a
  random deep transformer mixes tokens toward a shared sentence state, and training undoes that.
- With E01, this is the second time I over-estimated how "simple" (n-gram or broadcast) SONAR's trained states are.

## Implementation choices (see IMPLEMENTATION_NOTES.md)
- Models fit on content tokens only.
- Leave-own-sentence-out lookup for training rows.
- Q5 is scored as the best rank-r projection of the gain.
- B3 uses the nearest length for 14 of 8,000 sentences that had no same-length partner.
- **Unregistered extras:**
  - s alone (with position/length) gives .075;
  - the bag adds .002 over B1;
  - B2 gives .525.

## Claim tier
- **T2:** "trained SONAR final token states are not own-token + sentence broadcast. A random-init twin is, at .97."
- **T0:** anything about what the token-specific residual encodes.