# RESULT — Campaign100 036 norm-semantics

**Question.** What does the L2 norm ‖z‖ of a SONAR sentence embedding encode — length,
specificity/information-content, confidence, typicality, or nothing (a scale artifact)? And is
‖z‖ a *usable* semantic control (scale z ⇒ systematic decode change) or just magnitude?

**Verdict. ★ MIXED / mostly-NEGATIVE-on-control, POSITIVE-on-specificity, with a decisive
scale-invariance result and one important instrument caveat.** ‖z‖ carries a modest, genuine
**length + specificity/information-content** signal (longer, rarer, more named-entity-dense
sentences → higher norm, and the specificity part *survives* length control), but it does **NOT**
encode reconstruction difficulty or fabrication (confidence), and it is **NOT a usable semantic
control**: scaling z by ±30% is almost invisible to the decoder and to a human-proxy judge — the
SONAR decoder reads meaning from the **direction** of z, not its magnitude. The eye-catching
"norm↔density/intrinsic-dim" correlations are ~90 % a **mechanical artifact** of raw-Euclidean
distance and collapse in cosine space.

Tier: **T3-exploratory** (single AE, one corpus family, greedy-only decode).

## What ran
- **Corpus A** (diverse, n=29,029; 031's z.npy) — ‖z‖ vs token length, raw-Euclidean kNN density
  (mean dist to 20 NN), per-point MLE local intrinsic dim; by domain/length bin.
- **Corpus B** (021's frozen 1500; z = 027-roundtrip z0[:1500], verified aligned to 021 ids
  0–1499) — ‖z‖ vs length, 028 per-sentence reconstruction cosine (AE round-trip & SAE), 021
  fabrication labels, 028 local density, SAE effective-atoms/entropy, gpt2 perplexity, and a
  named-entity/number heuristic. Pearson + Spearman + **length-control partial correlations**.
- **Causal scaling probe (GPU0)** — 150 sentences (50/50/50 across length bins) × scale factors
  {0.7, 0.85, 1.0, 1.15, 1.3} = 750 deterministic greedy decodes (035 path); per factor:
  output length, chrF-vs-orig, re-encode cosine-to-original-direction, exact-match, gpt2-ppl,
  TTR; LOCAL codex neutral 3-way judge (length / specificity / fluency; NO attack vocab) on
  0.7/1.0/1.3 × 30.
- **Distribution** — CV, range, percentiles per corpus and length bin.

## Findings

### 1. Distribution — ‖z‖ varies modestly, not constant, not wildly
- Corpus A: mean 0.2245, **CV 0.156**, range 0.132–0.361 (ratio 2.7×).
- Corpus B: mean 0.1995, **CV 0.133**, range 0.137–0.287 (ratio 2.1×).
SONAR is *not* norm-constant, but the variation is modest (~13–16 %). "Norm carries little as a
semantic axis" is the right reading — see §3/§4.

### 2. What ‖z‖ correlates with (and a big caveat)
| correlate | corpus | Pearson r | partial r (ctrl length) |
|---|---|---|---|
| token length | A | **+0.490** | — |
| token length | B | +0.269 | — |
| gpt2 log-perplexity (specificity) | B | +0.336 | **+0.406** |
| named-entity/number count (specificity) | B | +0.336 | **+0.322** |
| SAE effective-atoms / entropy | B | +0.27 | +0.12 |
| reconstruction cosine (AE round-trip) | B | −0.084 | −0.065 |
| fabrication label | B | +0.064 | +0.033 |
| raw-Euclidean kNN density | A | **+0.876** | +0.843 |
| local intrinsic dim (MLE) | A | +0.501 | +0.319 |

- **Length: real, moderate.** Longer → higher norm (A r=0.49 > 0.30; B r=0.27). Monotone by bin
  (A short 0.205 < med 0.222 < long 0.244).
- **Specificity BEYOND length: real.** gpt2-perplexity partial r **+0.41** and NE-count partial
  **+0.32** (both *rise* after controlling length; ppl is even slightly negatively correlated
  with length, r=−0.18). Illustrative extremes (corpus B):
  - lowest-norm = generic/predictable: *"As the years went by, the industry began to change."*
    (ppl 13), *"We're looking forward to seeing him make an impact."* (ppl 18)
  - highest-norm = entity-dense/surprising: *"Ernst Wilhelm Nay: Kunst und Politik."*,
    *"Lovecraft, EC Comics, and classic Universal monster movies."* (ppl 352),
    *"…Marché aux Puces de Saint-Ouen, a famous flea market…"*
  So ‖z‖ does track **information density** (rare words, proper nouns, surprising content), not
  just length. This is the one clearly positive semantic content of the norm.
- **NOT confidence / difficulty.** ‖z‖ is ~uncorrelated with reconstruction quality (r=−0.08)
  and fabrication (r=+0.06). The 028 "difficulty axis" (length↔recon) does **not** live in the
  norm. (SAE-recon cosine r=−0.23 is entirely length-mediated → partial −0.05.)
- **★ Caveat — density / intrinsic-dim correlations are mostly a raw-distance artifact.** The
  headline norm↔density r=0.88 (corpus A) uses **raw-Euclidean** kNN distance, which is
  mechanically ∝ ‖z‖ (bigger vectors sit farther apart at equal cosine). In **cosine (unit-sphere)
  space** the correlation **collapses to r=0.21**, and norm vs (raw-density ÷ norm) ≈ **−0.09**.
  Because the MLE local-ID is likewise computed from raw-Euclidean kNN, the norm↔local-ID r=0.50
  inherits the same confound. **Interpretation: ‖z‖ does not meaningfully encode manifold density
  or local intrinsic dimension once the metric is made scale-invariant.** (Same withdrawn-claim
  lesson as 031's ID↔density entanglement.)

### 3. Causal scaling — ‖z‖ is NOT a usable semantic control (decoder is ~scale-invariant)
Scaling z by {0.7 … 1.3} produces essentially **no** systematic decode change:

| factor | out/orig wordlen | chrF vs orig | cos-dir orig | median ppl | exact | garble |
|---|---|---|---|---|---|---|
| 0.70 | 1.01 | 93.3 | 0.987 | 29.9 | 0.51 | 0.00 |
| 0.85 | 1.00 | 94.6 | 0.991 | 28.6 | 0.58 | 0.00 |
| 1.00 | 1.00 | 94.7 | 0.992 | 28.6 | 0.59 | 0.00 |
| 1.15 | 1.00 | 94.7 | 0.992 | 28.8 | 0.59 | 0.00 |
| 1.30 | 1.00 | 94.5 | 0.991 | 29.2 | 0.58 | 0.00 |

No monotone effect on length, genericness, or direction; the only signal is a **mild, symmetric,
non-monotone degradation** at the extremes (shrinking to 0.7 is slightly worse than growing to
1.3: chrF 93.3 vs 94.5, exact 0.51 vs 0.58). Example — identical decode at all five factors:
*"The linguistic relativity of conceptual metaphor."* → same string ×5. **LOCAL codex judge
(90 items, 0 parse fails):** length "same" 29–30/30 and specificity "same" **30/30 at every
factor**; fluency 27–29/30 fluent with garble counts unrelated to factor. A ±30 % norm change is
invisible to both the decoder and the human-proxy judge → **the SONAR decoder reads meaning from
the direction of z; magnitude is close to a free parameter over this range.**

### 4. Synthesis
‖z‖ is best described as a **weak length + information-density gauge**, not a control knob and not
a confidence/difficulty signal. Its variation (CV ~0.13–0.16) is modest and largely decorative
for downstream decoding: because the decoder is scale-invariant over ±30 %, the norm is closer to
a **near-normalization artifact with a faint specificity tint** than to a semantic axis.

## Predictions & Brier
Frozen in PREREG_LITE (correlation values + all decodes unseen; **(d) marginal CV disclosed-peeked
at plumbing** — reported separately):
- **(a)** ‖z‖↔length |r|>0.3 (corpus A): p=0.55 → **TRUE** (r=0.490).
- **(b)** specificity partial |r|>0.2 beyond length: p=0.35 → **TRUE** (ppl 0.41, NE 0.32).
  *Under-predicted — I over-weighted the "norm is mostly length/artifact" prior; the info-content
  component is genuine.*
- **(c)** scaling = usable semantic control: p=0.35 → **FALSE** (no monotone/interpretable effect;
  judge sees no change). *Correctly leaned <0.5.*
- **(d)** CV<0.15 on the 021-1500 corpus: p=0.85 → **TRUE** (CV 0.133) — **peeked, disclosed.**
- secondary **(c1)** any monotone metric effect: p=0.75 → **FALSE** (effect is symmetric, not
  monotone). *Miss — degradation is U-shaped around 1.0, not directional.*

**Brier (fair, a/b/c only) = 0.2492.** Including the peeked (d) and (c1): 0.2665. Dominant penalty
= the (b) under-prediction (specificity survived length control) and the (c1) miss (symmetric, not
monotone, degradation).

## Gates
- **G_norm PASS** — all ‖z‖ finite & >0 (A ∈[0.132,0.361], B ∈[0.137,0.287]).
- **G_det PASS (self)** — factor-1.0 decode reproducible **1.000** (batch-96 vs batch-1); factor-1.0
  exact-reconstruct 0.59 ≈ 021's 0.573 headline (pipeline valid). *Cross-method match to 021's
  `greedy` field is only 0.24 — a tokenizer/decoding-config difference (021 used the SONAR pipeline
  default, this uses raw argmax greedy per the 022/035 path); the self-determinism gate is the
  binding one and passes perfectly.*
- **G_length_partial PASS** — length-control partial correlation reported for every non-length
  correlate (the confound is controlled, not assumed away); it is what flips SAE-recon (−0.23→−0.05)
  and confirms specificity (ppl 0.34→0.41).
- **G_judge PASS** — 90/90 codex parse-ok, 0 fails.
- **G_power PASS** — n_A=29,029, n_B=1,500, scale 150×5=750 decodes.

## Limitations
Single AE (SONAR basic) / one corpus family (021+031 English) / greedy-only decode; scaling tested
only over ±30 % (larger factors may break down — not probed, deliberately, to isolate the
"usable control" question). Specificity proxies are gpt2-ppl + a capitalization/digit NE heuristic
(no spacy/wordfreq in venv) — directionally sound but coarse. The density/ID artifact caveat means
any *raw-Euclidean* geometry result touching norm should be re-checked in cosine space.

## Follow-up worth funding? **Weak-Y (narrow).**
- The norm↔specificity link is the only positive: worth a clean confirm with proper NE tagging +
  a controlled rare-word-injection stimulus (does adding a rare proper noun raise ‖z‖ at fixed
  length?) — connects 066 bits-accounting (norm as a crude bits gauge) and 070/071 numeric/entity
  fidelity.
- The scale-invariance result is a useful **negative primitive** for steering work (080/082/086):
  attacks/edits should move *direction*, not magnitude; a magnitude-only perturbation is a no-op.
- Lesson logged for the geometry block (037/039/043): raw-Euclidean kNN distance is norm-confounded
  — use cosine-space density before attributing anything to "density" or "intrinsic dim."