# RESULT — 045 k-atom-curve: inference-time top-m sweep on the fixed k=32 SAE

**Tier: T3-exploratory.** GPU0/phys0, tmux c100_045, full chain 4 min (analyze 3 s, decode 235 s),
SELF-HARVESTED in-session 2026-08-03. **Verdict: ★ POSITIVE with a twist — the FVU-vs-m curve is
U-SHAPED (best at m=64, m=128 WORSE than m=32 on both corpora) while decode chrF rises MONOTONICALLY
through m=128 (21.0 → 35.4): tail atoms carry decode-relevant content that L2 reconstruction
mis-prices. No saturation anywhere: even 128 atoms leave a 48-chrF gap to the x ceiling (83.8) —
"the sentence" does not survive inside this dictionary at any m. Atom mass is extremely diffuse
(median 2450 positive ReLU atoms; n50 ≈ 513, n90 ≈ 1505 — the trained k=32 cut keeps a sliver of
ranked mass), and atoms-needed is a structural constant: essentially UNCORRELATED with sentence
length (r = 0.04) or per-sentence difficulty (r = −0.11). Brier 0.1456 (P1/P2/P3 TRUE, P4 FALSE
on the correlation half).**

## What actually ran
Per PREREG_LITE.md, everything, nothing truncated. Fixed SAE = w40
`cpool-sae_h16384_k32_btk_e40_seed0.pt`, 028 loader (bf16→fp32, per-sample top-k, strict=False,
(x−x_mean)/scale). No training, no re-encode: pile val = 042 `data/full/z_val.npy` (4000 ok rows)
+ texts; corpus A = 027 `z.npy` first 12k + toklen. Inference-time m ∈ {1,2,4,8,16,32,64,128}
(top-m of ReLU pre-acts; m>32 = off-training-distribution, caveat stands). Random-from-top-32
control (true act values, 3 draws) at m ∈ {1,2,4,8,16} for FVU, m ∈ {2,8,16} decoded. Decode:
1000 pile-val sentences × 12 conditions, greedy beam=1 max_seq_len=96 (042 machinery), chrF +
exact + paired-bootstrap deltas vs M32. Smoke passed first (35 s). Box out/ has DONE; artifacts
pulled to repo out/; session c100_045 ended clean, GPUs left idle (35/15/15 MiB).

## Gates (all PASS)
- **G1 in-regime**: FVU(m=32, pile 4k) **0.7312** ∈ [0.70,0.76] — reproduces 042's 0.731 to 3
  decimals; deterministic (max|Δ| = 0.0); L0 = 32 by construction.
- **G2 decode validity**: chrF(x) **83.79** ≥ 50, headroom vs M32 **62.81** [61.59,63.92] ≥ 5.
- **G3 corpus-A cross-check**: FVU(m=32) **0.6822** ∈ [0.62,0.70] (043's stratified 12k gave 0.655;
  our slice is first-12k, band pre-widened for that).
- **G4 internal consistency**: rand-32-of-32 ≡ top-32 (ΔFVU = 0.0).

## (a) FVU vs m — U-shaped, minimum at m=64
| m | 1 | 2 | 4 | 8 | 16 | 32 | 64 | 128 |
|---|---|---|---|---|---|---|---|---|
| pile val | 1.074 | 1.027 | 0.966 | 0.895 | 0.815 | **0.731** | **0.687** | **0.825** |
| corpus A | 1.101 | 1.031 | 0.944 | 0.852 | 0.762 | **0.682** | **0.667** | **0.893** |
| rand-m (pile, 3-draw range) | 1.132–1.133 | 1.115 | 1.080–1.081 | 1.015–1.016 | 0.900–0.901 | ≡0.731 | — | — |

- m=1 is worse than predicting the mean (FVU > 1). Doubling k at inference buys only ~0.045 FVU
  (0.731→0.687); quadrupling BLOWS UP (+0.09 vs m=32 on pile, +0.21 on corpus A) — rank-33-128
  pre-acts, never trained to fire, add badly-scaled mass. P1 confirmed the off-dictionary reading:
  no inference-time m approaches the ckpt's own-corpus 0.44.
- Ranking matters for FVU at every m: top-m < min(rand-m draws) at m = 1,2,4,8,16 (P3 FVU half).

## (b) decode chrF vs m — monotone rise, no saturation, dissociated from FVU
| cond | x | M1 | M2 | M4 | M8 | M16 | M32 | M64 | M128 | R2 | R8 | R16 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| chrF | 83.8 | 10.0 | 10.7 | 12.7 | 14.6 | 17.0 | 21.0 | 28.3 | 35.4 | 11.5 | 12.6 | 14.6 |
| exact | .240 | 0 | 0 | 0 | 0 | 0 | .002 | .008 | .017 | 0 | 0 | 0 |

- Deltas vs M32 (paired bootstrap 95%): M64 **+7.28** [6.60,7.99], M128 **+14.41** [13.45,15.37];
  M16 −3.99, M8 −6.40, M2 −10.25, M1 −10.94. Every step of the ladder is significant.
- **The dissociation is the headline**: from m=64→128 FVU worsens by +0.14 while chrF improves by
  +7.1. The decoder extracts content from tail atoms whose L2 contribution is net harmful. Not a
  norm artifact per 042 (matched-norm Gaussian LOSES 4.4 chrF there), but mechanism unresolved and
  m>32 is off-distribution — flagged, not over-claimed.
- chrF(M128) = 35.4 << 50 << x 83.8: even at 4×k the sentence does not "survive" — consistent with
  042/043/044 (most decodable content is off-dictionary; width didn't help, and inference-k only
  crawls toward it).
- Ranking caveat (honest): **RAND2 slightly beats M2 on chrF** (11.47 vs 10.73; deltas vs M32
  −9.52 vs −10.25, CIs overlap but ordering is consistent) — the top-ranked 1-2 atoms are
  reconstruction/scaffold-oriented, while a random pair from the firing set samples more diverse
  content. At m=8/16 top-m clearly wins (14.59 vs 12.62; 17.00 vs 14.60). P3's frozen criterion
  (m=8) TRUE.

## (c) atoms-per-sentence — diffuse and structurally constant
- Full-ReLU mass (pile / corpus A): median L0_relu **2450 / 2465** of 16384 positive; n50 median
  **513 / 512** (p10-p90: 371-687); n90 median **1505 / 1512** (p10-p90: 1107-1990). The trained
  k=32 keeps ≈6% of the atoms that carry 50% of ℓ1 mass — the encoder spreads mass very broadly.
- **Correlations are ~zero**: corr(n90, wordcount) = **−0.02** (pile), corr(n90, toklen) = **+0.04**
  (corpus A), corr(n90, per-sentence FVU@32) = **−0.11**. Atoms-needed does NOT track length or
  difficulty (028's axis does not extend here) — concentration looks like a property of the
  encoder/dictionary, not of the sentence. (This killed P4's joint.)

## (d) qualitative ladder (25 samples in out/decode_samples.json, unscored)
What appears first is a **generic sentence scaffold, then topic, then specifics**:
- m=1: near-content-free frames — "The history of", "My", "I." (often short).
- m=2-4: topic + syntactic frame, wrong content ("I don't play games to me" for a gaming comment).
- m=8-16: gist emerges ("I know that you start in the 10th and I started in the 11th" for a Spanish
  counter-at-11 sentence — note cross-lingual gist arriving before wording).
- m=32: recognizable paraphrase with mangled entities ("Aberdeen's presence is coming to Mindsong").
- m=64-128: named entities and specifics lock in ("Aberfberth finds … meeting in Mundingus in
  Hogsmeade"; "I'd set the counter at 11th"). Entities/numbers are the LAST thing the dictionary
  provides — matching r-carries-BoW/lexical-content from 043.

## Brier vs frozen predictions
- **P1 (0.80)** FVU(m=128) > 0.45 → **TRUE** (0.825). Brier 0.04. (Right direction, though the
  prediction under-imagined it: not just short of 0.45 but WORSE than m=32.)
- **P2 (0.60)** chrF(M128) ≥ chrF(M32)+5 AND < 50 → **TRUE** (35.4 vs 21.0+5). Brier 0.16.
- **P3 (0.85)** top-m beats rand-m: FVU all m ∈ {1..16} AND chrF at m=8 → **TRUE**. Brier 0.0225.
- **P4 (0.60)** n90 median ≥ 64 AND corr(n90, toklen) ≥ 0.20 → **FALSE** (1505 ✓ but r = 0.04 ✗).
  Brier 0.36. First half was trivially true, second half decisively false — the informative miss.
- **Mean Brier 0.1456.**

## Limitations
- m > 32 cells are off-training-distribution for a BatchTopK-trained SAE (the trained model never
  used rank-33+ atoms); the chrF gains there describe THIS inference procedure, not a trained k=64/128
  SAE. A retrained-k sweep (true k-ladder) is the clean version — not run here.
- Single SAE (h16384 seed0), single decode corpus (pile val), greedy decode only; random control
  restricted to the top-32 firing set (whole-dictionary random not tested); n50/n90 defined on full
  ReLU pre-acts (encoder-specific notion of "mass").
- Qualitative section is observational, not judged/scored. Everything T3-exploratory.

## Follow-up worth funding? **Y**
Two forks: (1) TRAINED k-ladder (k ∈ {64,128,256} same recipe, ~min/train at h16384) — does trained
k=128 close the decode gap that inference-k cannot, or does FVU-vs-content dissociate there too?
(2) The FVU/chrF dissociation itself: what do rank-33-128 atoms add that decoders love but L2
hates? (candidates: low-norm content directions vs high-norm scaffold; connects 036 norm-semantics.)