# Appendix G — The L2 "organism" text autoencoders: training and generalisation

Companion page to the [manuscript](index.html) appendix plan · written 12 September 2026 · all numbers are read from archived files; no model was loaded and no training was run.

> **Scope.** This page documents how the small trained text autoencoders used as L2 substrates (PLAN_V2 §L2; experiment B2) were built, how much data they saw, and what "generalisation" means for them. It is an appendix: it supports the B2 rows of the shortfall table and the case-study provenance. It does not add a benchmark result. Every number carries its source path. Numbers computed on this box for this page (the corpus token counts of G.5) are marked as such and are reproducible from the listed script.

## G.1 Lineage and naming [existing result]

| name | objective | date | status | source |
|---|---|---|---|---|
| pilot v0 | surface → surface reconstruction; 3 syntactic frames | 2026-07-02 | superseded; 7,542,784 params, 58,765 steps, 194,730,362 tokens, 45 min, exact 1.000 | `/data/research/benchmarks/night8/organism/PILOT_REPORT.md`, `organism/pilot_results/metrics.json` |
| **organism_v1** (this page) | surface paraphrase → **canonical** rendering of the meaning tuple | 2026-07-02 | the L2 substrate used in B2; d_z ∈ {256, 64}, seed 0, plus three d_z=64 seed replicates | `/data/research/benchmarks/night8/organism_v1/V1_REPORT.md`; seeds `/data/research/benchmarks/night8/organism_v1_seeds/SEEDS_REPORT.md` |
| organism_v2 | two-clause surface → **computed** canonical (sum of counts, sorted agents, categories) | 2026-07-02 | comparison rows only (G.4, G.6.4) | `/data/research/benchmarks/night8/organism_v2/V2_REPORT.md` |

<figure class="explainer organism-comparison" aria-labelledby="organism-comparison-caption">
  <div class="organism-card-grid">
    <article class="organism-card">
      <header><span class="diagram-kicker">pilot v0 · copy surface</span><strong>Reconstruction</strong></header>
      <div class="scale-metrics"><span><b>616</b><small>d<sub>vocab</sub> ids</small></span><span><b>11.17</b><small>mean input tokens</small></span><span><b>13</b><small>max input tokens</small></span><span><b>256</b><small>d<sub>z</sub></small></span></div>
      <div class="score-row"><span>Held-out exact reconstruction</span><b>1.000</b></div>
      <div class="score-track" aria-hidden="true"><i style="--score:100%"></i></div>
      <small class="score-note">Also 1.000 on the decorrelated set.</small>
    </article>
    <article class="organism-card featured-card">
      <header><span class="diagram-kicker">organism_v1 · paraphrase → canonical</span><strong>Canonicalisation</strong></header>
      <div class="scale-metrics"><span><b>957</b><small>d<sub>vocab</sub> ids</small></span><span><b>9.80</b><small>mean input tokens</small></span><span><b>18</b><small>max input tokens</small></span><span><b>64 / 256</b><small>d<sub>z</sub></small></span></div>
      <div class="score-row"><span>Held-out-combo exact match</span><b>1.000</b></div>
      <div class="score-track" aria-hidden="true"><i style="--score:100%"></i></div>
      <small class="score-note">Both bottlenecks; in-distribution and decorrelated are also 1.000.</small>
    </article>
    <article class="organism-card warning-card">
      <header><span class="diagram-kicker">organism_v2 · computed canonical</span><strong>Semantic computation</strong></header>
      <div class="scale-metrics"><span><b>735</b><small>d<sub>vocab</sub> ids</small></span><span><b>15.3</b><small>mean input tokens</small></span><span><b>23</b><small>max input tokens</small></span><span><b>64 / 256</b><small>d<sub>z</sub></small></span></div>
      <div class="score-row"><span>In-distribution exact match</span><b>1.000</b></div>
      <div class="score-track" aria-hidden="true"><i style="--score:100%"></i></div>
      <div class="score-row"><span>Held-out pair exact match</span><b>.683 / .685</b></div>
      <div class="score-track caution-track" aria-hidden="true"><i style="--score:68.5%"></i></div>
      <div class="score-row"><span>Held-out totals exact match</span><b>0.000</b></div>
      <div class="score-track fail-track" aria-hidden="true"><i style="--score:0%"></i></div>
      <small class="score-note">d<sub>z</sub>=64 / 256. The surface-reconstruction twin scores .995 on held-out pairs and 1.000 on held-out totals.</small>
    </article>
  </div>
  <figcaption id="organism-comparison-caption"><b>Organism family at a glance.</b> Input length and d<sub>vocab</sub> give the scale of each closed synthetic task. Exact match measures copying for pilot v0, canonicalisation for v1, and the computed target for v2, so the labels matter more than the superficially similar percentages.</figcaption>
</figure>

The seed replicates reuse `train_v1.py`, `corpus_v1.py`, `model.py` byte-for-byte (diff against `4l:/workspace/HOME/guest/night8/organism_v1/` checked 12 September 2026: identical). Checkpoints live only on the GPU box: `4l:/workspace/HOME/guest/night8/organism_v1/run_dz{256,64}/{ckpt_best,ckpt_final}.pt` and `4l:/workspace/HOME/guest/night8/organism_v1_seeds/run_seed{1,2,3}/ckpt_final.pt` (30 MB each, no optimiser state).

## G.2 Architecture and objective [existing result]

Source: `/data/research/benchmarks/night8/organism_v1/model.py`, `train_v1.py`.

| component | value |
|---|---|
| vocabulary | 957 ids = 953 content types (exhaustively enumerated from the generator; no UNK possible) + `<pad> <bos> <eos> <unk>`; whitespace tokenisation |
| token embedding | 256-d, tied to the output projection (`out.weight = tok_emb.weight`) |
| positions | learned, separate encoder/decoder tables, max_len 24 |
| encoder | 4 × `nn.TransformerEncoderLayer`, d_model 256, 4 heads, FFN 1024, GELU, dropout 0.1 |
| bottleneck | masked mean-pool over encoder states → `Linear(256 → d_z)` = z; `Linear(d_z → 256)` = m |
| decoder | 4 × `nn.TransformerDecoderLayer` (same widths); m is the length-1 cross-attention memory **and** is added to every decoder token embedding; causal mask; greedy decoding, max 14 tokens |
| parameters | d_z=256: 7,761,664 · d_z=64: 7,663,168 (`metrics.json:params`; count de-duplicates the tied weight) |
| sequence caps | surface ≤ 20 tokens (corpus max 18), canonical ≤ 13 (corpus max 11) |

**Objective.** Paraphrase-canonicalisation: encode a *surface* sentence, decode the fixed-order *canonical* rendering of the same meaning tuple — `AGENT [not] VERB[tense] [COUNT as digits] PATIENT ADJS [in the LOC] .` — with canonical synonyms and explicit `not`. Loss is token cross-entropy on the canonical stream (pad ignored, no label smoothing), teacher-forced. Surface ≠ canonical for 0.99368 of training pairs (V1_REPORT §1; recomputed here 0.993963, G.5). The canonical form has an exact inverse parser (`corpus_v1.parse`), which is what every "round-trip" number below uses.

<figure class="explainer architecture-diagram" aria-labelledby="architecture-caption">
  <div class="flow-row" role="img" aria-label="A surface paraphrase passes through a four-layer Transformer encoder, mean pooling, a 64- or 256-dimensional bottleneck, expansion to decoder memory, and a four-layer Transformer decoder to produce a canonical sentence.">
    <div class="flow-card example-card"><span class="diagram-kicker">surface input</span><strong>“In the park, three bright dogs were pursued by Ada.”</strong><small>variable wording · ≤ 20 tokens</small></div>
    <span class="flow-arrow" aria-hidden="true">→</span>
    <div class="flow-card"><span class="diagram-kicker">encode</span><strong>4 × Transformer</strong><small>256-d token states</small></div>
    <span class="flow-arrow" aria-hidden="true">→</span>
    <div class="flow-card bottleneck-card"><span class="diagram-kicker">compress</span><strong>mean pool → z</strong><small>64 or 256 numbers</small></div>
    <span class="flow-arrow" aria-hidden="true">→</span>
    <div class="flow-card"><span class="diagram-kicker">decode</span><strong>4 × Transformer</strong><small>z becomes memory m</small></div>
    <span class="flow-arrow" aria-hidden="true">→</span>
    <div class="flow-card example-card"><span class="diagram-kicker">canonical target</span><strong>“Ada pursued 3 dog bright in the park.”</strong><small>fixed order · ≤ 13 tokens</small></div>
  </div>
  <div class="objective-strip"><span><b>Training signal</b> token cross-entropy</span><span><b>At inference</b> causal greedy decoding</span><span><b>Round-trip check</b> exact inverse parser</span></div>
  <figcaption id="architecture-caption"><b>Figure G1.</b> The “organism” is a paraphrase canonicaliser, not a surface-copying autoencoder. Every input must pass through the single sentence vector <i>z</i>.</figcaption>
</figure>

**Optimiser and schedule** (`train_v1.py` defaults, unchanged in every run):

| setting | value |
|---|---|
| optimiser | AdamW, lr 3e-4, weight decay 0.01, gradient clip 1.0 |
| schedule | linear warm-up 500 steps, cosine to a 5 % floor over a 40,000-step horizon, then constant at the floor (1.5e-5) |
| batch | 256 pairs, reshuffled each pass over the 2,000,000 pre-tokenised pairs |
| budget | wall-clock: 90 min (seed 0 runs), 35 min (seed 1–3 replicates); no step target |
| evaluation | every 2,000 steps on three fixed sets: in-distribution 3,000 (seed 8801), held-out role combos 1,500 (seed 8802), decorrelated 1,500 (seed 8803) |
| checkpointing | `ckpt_best.pt` overwritten when in-distribution exact match strictly improves; `ckpt_final.pt` at the deadline |
| early stopping | **none in v1** — the run ends at the wall-clock deadline. v2 (`train_v2.py`) adds a rule: stop when in-distribution exact match has been 1.000 at every evaluation for 20 min (`--plateau_minutes 20`) |

Hardware: one RTX A4000 per run on the 4l box (NVML broken; device liveness verified by a trial matmul; memory read from `torch.cuda.max_memory_allocated`).

## G.3 Data-generating process [existing result]

Source: `/data/research/benchmarks/night8/organism_v1/corpus_v1.py` (constants and `sample_factors`), `ORGANISM_V1_SPEC.md`, V1_REPORT §1.

**Meaning factors** (all logged; all required by the canonical target):

| factor | cardinality | sampling |
|---|---|---|
| verb class | 4 (pursuit, social, handling, perception) | social p=.35, others .65/3 each |
| verb | 40 (10 per class) | Zipfian within class |
| agent | 100 names (one pool, 50 female + 50 male) | Zipfian |
| patient | 80 object concepts (8 categories × 10) **or** a person from the same 100-name pool (≠ agent) when the verb is social → two-person events, 0.34934 of the corpus | Zipfian |
| count | 2–99 (98 values), digits in the canonical form | present p=.6, object patients only |
| adjectives | 0–2 from 30, canonical-sorted | 0 / 1 / 2 with p .45 / .35 / .20; Zipfian |
| location | 40 (4 classes × 10) | present p=.5; class = verb class with p=.9 (`decorrelate=True` → uniform) |
| tense | past / present | uniform |
| negation | binary | p=.15 |

**String-inert factors** (logged, never rendered; theorem-grade absent): mood (3), narrator (20).

**Surface latents** (change the surface, not the target): paraphrase family (9: active .18, passive .15, cleft .10, fronted-locative .10 [needs a location], nominalisation .10, there-construction .10, question .04, topicalisation .13, pseudo-cleft .10), verb synonym (2–3 per concept), object synonym (2 per concept), number style (digit / word), filler adverb (8, p=.3), locative preposition (3).

**Corpus:** 2,000,000 (surface, canonical, factors) triples, `sample_pairs(2_000_000, seed=0, split="train")`, generated in-process (18.2 s on 4l, 19.5 s here). Unique surfaces 1,964,397 (`metrics.json:train_unique_surface`; recomputed identically in G.5). Surface length mean 9.80 / max 18; canonical mean 6.79 / max 11.

<figure class="explainer dgp-diagram" aria-labelledby="dgp-caption">
  <div class="dgp-source"><span class="diagram-kicker">sample once</span><strong>Meaning tuple</strong><small>agent · verb · patient · count · adjectives · location · tense · negation</small></div>
  <div class="branch-grid">
    <div class="branch-card surface-branch"><span class="branch-label">Surface branch</span><strong>9 paraphrase families</strong><p>Synonyms, number style, filler adverb and locative preposition change the wording.</p><span class="branch-output">many possible sentences</span></div>
    <div class="branch-card target-branch"><span class="branch-label">Target branch</span><strong>1 canonical renderer</strong><p>Fixed field order, canonical synonyms, digits and explicit negation.</p><span class="branch-output">one parseable target</span></div>
  </div>
  <div class="inert-note"><b>Logged but not rendered:</b> mood and narrator. They cannot be recovered from either string.</div>
  <div class="split-row" aria-label="Evaluation split comparison">
    <div><span class="split-dot train-dot"></span><strong>Train / in-distribution</strong><small>same generator and correlations</small></div>
    <div><span class="split-dot heldout-dot"></span><strong>Held-out roles</strong><small>new ordered verb–agent–patient triples</small></div>
    <div><span class="split-dot decorrelated-dot"></span><strong>Decorrelated</strong><small>verb ↔ location alignment broken</small></div>
  </div>
  <figcaption id="dgp-caption"><b>Figure G2.</b> One sampled meaning produces a variable surface sentence and a deterministic canonical target. The evaluation splits change combinations or correlations—not the vocabulary or grammar.</figcaption>
</figure>

**Held-out split rule.** An ordered two-person combination (verb, agent, patient) is held out iff the first byte of `md5("verb|agent|patient")` is divisible by 20 (≈ 5 % of ordered combos). `split="train"` never emits a held-out combination; `split="heldout"` emits only those. Leakage in both directions was asserted 0 on 20,000 / 2,000 samples at build time (V1_REPORT §1). Because the rule hashes the *ordered* triple, the reverse role assignment of a held-out combination may occur in training.

## G.4 Compute accounting per checkpoint [existing result]

Sources: `run_dz256/metrics.json`, `run_dz64/metrics.json` under `/data/research/benchmarks/night8/organism_v1/`; `figures/organisms/seed{1,2,3}_metrics.json` (verbatim copies of `4l:/workspace/HOME/guest/night8/organism_v1_seeds/run_seed{1,2,3}/metrics.json`, SHA-256 in `figures/organisms/SHA256SUMS`); `/data/research/benchmarks/night8/organism_v2/run_{sem_dz64,sem_dz256,recon_dz64}/metrics.json`. Derived columns: tokens/param = tokens_seen / params; epochs = steps × 256 / 2,000,000.

"Tokens seen" is `train_v1.py`'s counter (line 177): non-pad source tokens plus non-pad decoder-input tokens per pair. The decoder input is `<bos>` + canonical + `<eos>` with its last column dropped, so `<eos>` is counted for every item except the batch-longest ones. Per pair this is 18.57 tokens (544,050,648 / (114,440 × 256)); content tokens without `<bos>`/`<eos>` are 16.59 per pair (G.5).

| checkpoint | d_z | seed | params | steps | tokens seen | tokens / param | epochs | wall-clock | tokens / s | peak GPU MB | stop reason |
|---|---|---|---|---|---|---|---|---|---|---|---|
| organism_v1 `run_dz256` | 256 | 0 | 7,761,664 | 114,440 | 544,050,648 | 70.1 | 14.65 | 5,400 s | 100,749 | 803 | 90-min budget |
| organism_v1 `run_dz64` | 64 | 0 | 7,663,168 | 110,274 | 524,244,249 | 68.4 | 14.12 | 5,400 s | 97,082 | 802 | 90-min budget |
| organism_v1_seeds `run_seed1` | 64 | 1 | 7,663,168 | 45,628 | 216,914,120 | 28.3 | 5.84 | 2,100 s | 103,291 | 807 | 35-min budget |
| organism_v1_seeds `run_seed2` | 64 | 2 | 7,663,168 | 45,369 | 215,748,897 | 28.2 | 5.81 | 2,100 s | 102,737 | 809 | 35-min budget |
| organism_v1_seeds `run_seed3` | 64 | 3 | 7,663,168 | 45,273 | 215,223,299 | 28.1 | 5.79 | 2,100 s | 102,487 | 802 | 35-min budget |
| organism_v2 `run_sem_dz64` (comparison) | 64 | 0 | 7,608,384 | 54,000 | 382,082,514 | 50.2 | 6.91 | 3,131 s | 122,030 | 949 | 100 %-exact plateau 20 min |
| organism_v2 `run_sem_dz256` (comparison) | 256 | 0 | 7,706,880 | 52,000 | 367,935,409 | 47.7 | 6.66 | 2,997 s | 122,771 | 952 | 100 %-exact plateau 20 min |
| organism_v2 `run_recon_dz64` (surface-reconstruction twin) | 64 | 0 | 7,608,384 | 68,000 | 568,973,540 | 74.8 | 8.70 | 5,405 s | 105,273 | 1,219 | 90-min budget |

Time to ceiling (in-distribution exact match, evaluations every 2,000 steps; from `eval_log`): first ≥ 0.99 at step 10,000 / 8,000 / 10,000 / 10,000 / 10,000 and first 1.000 at step 26,000 / 26,000 / 28,000 / 26,000 / 30,000 for dz256, dz64, seed1, seed2, seed3. Small dips (≥ 0.9993) occur between the first 1.000 and step 40,000; after step 40,000 every evaluation on all three splits is 1.000 for all five runs (checked from the same files). The seed 0 runs therefore spent roughly 65 of their 90 minutes on a flat plateau; the 35-min replicates stop after about 10 min of plateau (SEEDS_REPORT §Setup). v2's rows are included because they are the only runs of this family with a rule-based stop; their tokens/param (48–50) is where the plateau rule cuts the same schedule.

## G.5 Non-repeated tokens per parameter [existing result]

Computed 12 September 2026 on this box by `/data/research/tae-interp/neuralese-paper/corpus_token_stats.py` (CPU only; no torch; the `Vocab` class is executed from the archived `model.py`), output `figures/organisms/corpus_token_stats.json`, log `figures/organisms/corpus_token_stats.log`. The corpus is regenerated with the archived `corpus_v1.sample_pairs(2_000_000, seed, split="train")`, the exact call in `train_v1.py`. Check: the recomputed unique-surface count for seed 0 equals the logged 1,964,397; seeds 1–3 reproduce their logged 1,964,165 / 1,964,356 / 1,963,945. Generation took about 20 s per seed; no sub-sampling was needed.

**(a) Tokens in the de-duplicated corpus, seed 0.** Every surface string maps to exactly one canonical string, so unique (surface, canonical) pairs = unique surfaces = 1,964,397 (unique canonicals: 1,702,567 — different paraphrases of the same meaning tuple share a target).

| quantity (seed 0 corpus) | tokens | per param, d_z=256 (7,761,664) | per param, d_z=64 (7,663,168) |
|---|---|---|---|
| one pass over the 2,000,000 pairs, content tokens (surface + canonical) | 33,175,564 | 4.27 | 4.33 |
| one pass, `train_v1.py` accounting (+ `<bos>` + `<eos>` per pair; upper bound) | 37,175,564 | 4.79 | 4.85 |
| **unique (surface, canonical) pairs, content tokens** | **32,786,808** | **4.22** | **4.28** |
| unique pairs, `train_v1.py` accounting | 36,715,602 | 4.73 | 4.79 |
| **(b) unique surface strings, source tokens only** | **19,361,168** | **2.49** | **2.53** |

Consistency check: one pass under the `train_v1.py` accounting (37,175,564) × 14.648 passes = 544.6 M, against the logged 544,050,648 tokens seen (difference 0.1 %, the dropped `<eos>` of the batch-longest items).

**Tokens seen vs. non-repeated tokens.**

| checkpoint | tokens seen / param | unique-pair tokens / param (`train_v1` accounting) | ratio (≈ passes over the unique corpus) | epochs from steps |
|---|---|---|---|---|
| dz256 seed 0 | 70.1 | 4.73 | 14.8 | 14.65 |
| dz64 seed 0 | 68.4 | 4.79 | 14.3 | 14.12 |
| dz64 seed 1 | 28.3 | 4.79 | 5.91 | 5.84 |
| dz64 seed 2 | 28.2 | 4.79 | 5.87 | 5.81 |
| dz64 seed 3 | 28.1 | 4.79 | 5.86 | 5.79 |

The 70 tokens/parameter headline is therefore about 4.7 distinct tokens per parameter seen roughly 15 times each; the replicates saw the same 4.7 distinct tokens about 6 times each and reach the same 1.000 finals (G.6).

<figure class="explainer exposure-diagram" aria-labelledby="exposure-caption">
  <div class="exposure-equation" role="img" aria-label="For the d z 256 seed zero run, 4.73 unique-pair tokens per parameter repeated about 14.8 times equals 70.1 tokens seen per parameter.">
    <div class="metric-block"><strong>4.73</strong><span>unique-pair tokens<br>per parameter</span></div>
    <span class="math-mark" aria-hidden="true">×</span>
    <div class="passes-block"><div class="pass-tiles" aria-hidden="true"><i></i><i></i><i></i><i></i><i></i><i></i><i></i><i></i><i></i><i></i><i></i><i></i><i></i><i></i><i class="partial"></i></div><strong>14.8 passes</strong><span>over nearly the same evidence</span></div>
    <span class="math-mark" aria-hidden="true">≈</span>
    <div class="metric-block result-block"><strong>70.1</strong><span>tokens seen<br>per parameter</span></div>
  </div>
  <div class="exposure-compare"><span><b>90-min seed 0</b> ≈ 15 passes</span><span><b>35-min replicates</b> ≈ 6 passes</span><span><b>Behavioural final</b> 1.000 for both</span></div>
  <figcaption id="exposure-caption"><b>Figure G3.</b> “Tokens seen” counts repeated exposure. The distinct evidence budget is about 4.7 tokens per parameter; longer runs revisit it more often.</figcaption>
</figure>

**(c) Distinct token n-grams, seed 0 corpus** (a diversity measure; counted within sequences after the 20/13 caps):

| n | surface (source) | canonical (target) | either |
|---|---|---|---|
| 1 | 953 | 432 | 953 |
| 2 | 131,187 | 35,034 | 141,893 |
| 3 | 1,448,575 | 968,815 | 2,354,793 |
| 4 | 3,872,236 | 2,442,890 | 6,278,920 |

For scale: the source side has 0.50 distinct 4-grams per parameter (d_z=256), and the whole canonical side uses 432 of the 953 content types (names, canonical lemmas in two tenses, digits, canonical nouns, adjectives, locations, `not`, `in`, `the`, `.`).

**Seeds 1–3** (same generator, `seed=1,2,3`; the seed also sets weight init and batch order, SEEDS_REPORT §Setup):

| seed | unique surfaces | unique pairs, content tokens | unique surface, source tokens | distinct source 4-grams |
|---|---|---|---|---|
| 1 | 1,964,165 | 32,783,930 | 19,359,409 | 3,876,201 |
| 2 | 1,964,356 | 32,797,637 | 19,370,573 | 3,872,341 |
| 3 | 1,963,945 | 32,781,640 | 19,360,739 | 3,875,127 |

## G.6 Generalisation: what was measured and what it means [existing result]

**Definition used on this page.** For these models "generalisation" means correct canonicalisation of sentences whose *factor combination* was not in the 2,000,000 training pairs, inside a closed synthetic vocabulary of 953 types and nine paraphrase families. It is not transfer to open-domain text, to unseen words, or to unseen constructions. Every split below is drawn from the same generator; only the combination rule differs.

### G.6.1 organism_v1: the three training-time splits

Sources: `run_dz{256,64}/metrics.json:final`, `summary_tables.md`, V1_REPORT §2; seeds from `figures/organisms/seed{1,2,3}_metrics.json:final`.

| split | what is new relative to training | n | exact canonical match, dz256 / dz64 / seed1 / seed2 / seed3 | parse-fail | role-binding accuracy (two-person items) |
|---|---|---|---|---|---|
| in-distribution | fresh sample from the training distribution (seed 8801); string collision rate with the training set was **not measured** | 3,000 | 1.000 / 1.000 / 1.000 / 1.000 / 1.000 | 0.000 all | 1.000 all (n=1,017) |
| held-out role combos | ordered (verb, agent, patient) two-person triples never sampled in training (md5 rule, §G.3) | 1,500 | 1.000 / 1.000 / 1.000 / 1.000 / 1.000 | 0.000 all | 1.000 all (n=1,500) |
| decorrelated | location class drawn uniformly, breaking the p=.9 verb-class ↔ location-class alignment of training | 1,500 | 1.000 / 1.000 / 1.000 / 1.000 / 1.000 | 0.000 all | 1.000 all (n=525) |

Per-factor round-trip (parse of the greedy canonical output vs. logged factors) is 1.000 for all eight meaning factors — verb, agent, patient, count, adjs, location, tense, negation — on all three splits, for all five checkpoints (`final.*.per_factor` in each `metrics.json`; `summary_tables.md` "Per-meaning-factor round-trip"). Token accuracy is 1.000 on every final split.

The held-out-combo split is the compositional certificate of the v1 report: those exact role assignments were never seen, yet agent and patient are placed correctly in the canonical order. Its limits: the reverse assignment may have been seen; every name, verb and construction was seen many times; and the split is 5 % of two-person triples, not a held-out name or verb.

A weaker out-of-DGP datum exists as a by-product of the M2b battery (V1_REPORT §3, M2b note): verb toggles applied to unconstrained backgrounds create verb-class × patient-type combinations that the generator never produces (e.g. a social verb with a counted object patient); base round-trip on those sentences is 0.961 (dz256) and 0.977 (dz64), versus 1.000 on in-DGP backgrounds. This is the only measurement of behaviour outside the generator's support, and it was not designed as one.

### G.6.2 Seed replication (d_z=64, seeds 1–3, 35 min)

Source: `/data/research/benchmarks/night8/organism_v1_seeds/SEEDS_REPORT.md` and the seed `metrics.json` files. All three replicates reach 1.000 on every split, factor and binding measure with 5.8 epochs (G.4). What varies across seeds is not the behavioural ceiling but the certification *tier* of weak surface latents (loc_prep, num_style, patient_syn, verb_syn flip between P_NONLINEAR_ONLY / P_LINEAR / NO_GT; SEEDS_REPORT §1); the eight meaning factors and binding are P_LINEAR in all seeds.

### G.6.3 B2: agent-disjoint TEST items (the paper's substrate check)

Source: `/data/research/tae-interp/neuralese-paper/experiments/B2-organism-shortfall/results/TABLE.md` (coverage lines and READ tables per tag; "Seed spread" section), `results/read.json`, `NOTES.md` (stimulus design). Stimuli: the 100 names are shuffled with seed 20260912; TRAIN uses the first 60, TEST the last 40; two-person items draw both roles from the split's names; 300 TEST items per factor; backgrounds sampled with `split="any"`, so 46 of the 300 ROLE TEST items fall on training-held-out (verb, agent, patient) combos. "Certified" = the decoder parses both the base and the counterfactual item to their logged factors (`parse(D(z_src))` and `parse(D(z_tgt))` exact). "Lens" = accuracy of reading the factor off the decoder's own output; it is the READ ceiling.

| checkpoint | coverage: ROLE / POLARITY / TENSE / COUNT / CONTENT (of 300 each) | canonical re-encode round-trip | lens (READ ceiling): role_id / POL / TENSE / COUNT / CONTENT |
|---|---|---|---|
| dz256 (`ckpt_best`) | 0.990 / 1.000 / 0.993 / 0.993 / 0.997 | 0.970 | 0.995 / 1.000 / 1.000 / 1.000 / 0.998 |
| dz64 (`ckpt_best`) | 1.000 / 1.000 / 0.997 / 0.997 / 1.000 | 0.983 | 1.000 / 1.000 / 1.000 / 1.000 / 1.000 |
| dz64 seed 1 (`ckpt_final`) | 1.000 / 1.000 / 1.000 / 1.000 / 1.000 | 0.997 | 1.000 / 1.000 / 1.000 / 1.000 / 1.000 |
| dz64 seed 2 | 1.000 / 1.000 / 1.000 / 1.000 / 1.000 | 0.997 | 1.000 / 1.000 / 1.000 / 1.000 / 1.000 |
| dz64 seed 3 | 1.000 / 1.000 / 1.000 / 1.000 / 1.000 | 0.990 | 1.000 / 1.000 / 1.000 / 1.000 / 1.000 |
| untrained copy, d_z=256 (control) | 0.000 / 0.000 / 0.000 / 0.000 / 0.000 | 0.000 | 0.000 on every cell |
| untrained copy, d_z=64 (control) | 0.000 / 0.000 / 0.000 / 0.000 / 0.000 | 0.000 | 0.000 on every cell |

Reading: on names the models never saw in the TRAIN half of the stimulus set (but did see in pre-training) the decoder certifies 297–300 of 300 items per factor, and reading each factor off the decoder output is at ceiling (0.995–1.000). The untrained same-architecture copies certify 0 items and have a lens of 0.000 (their decodes never parse), which is the intended null: coverage is a property of training, not of the parser. Note the asymmetry inside the untrained rows: linear probes on the untrained z still read POLARITY at 0.998 / 0.990 and COUNT at 0.768 / 0.403 (`TABLE.md`, untrained READ), so "readable from z" and "certified through the decoder" are different endpoints, as the manuscript's editorial commitment 5 requires.

Seed spread of the READ cells (fraction of lens; four d_z=64 models: seed 0 at 90 min and seeds 1–3 at 35 min; `TABLE.md` "Seed spread"):

| cell (linear probe) | dz64 | seed 1 | seed 2 | seed 3 | mean ± sd [min, max] |
|---|---|---|---|---|---|
| ROLE:role_id | 1.00 | 1.00 | 1.00 | 1.00 | 1.000 ± 0.000 [1.00, 1.00] |
| POLARITY | 1.00 | 1.00 | 1.00 | 1.00 | 1.000 ± 0.000 [1.00, 1.00] |
| TENSE | 1.00 | 1.00 | 1.00 | 1.00 | 1.000 ± 0.001 [1.00, 1.00] |
| COUNT | 0.95 | 0.97 | 0.94 | 0.91 | 0.945 ± 0.023 [0.91, 0.97] |
| CONTENT | 0.99 | 1.00 | 0.99 | 0.99 | 0.995 ± 0.003 [0.99, 1.00] |

The 98-way COUNT read is the one cell with visible seed variance (0.91–0.97 of ceiling) and the one where a 35-min replicate is not systematically better or worse than the 90-min reference. ROLE-XC (train on active renderings, test on passives) stays at 1.000 for role_id across all four models; the binary `role_bin` label reads at chance (0.47–0.53) in every model — the TRACE caveat that a designated-A binary label confounds binding with a name-comparison nonlinearity (NOTES.md, READ cells).

### G.6.4 organism_v2 as the contrast: where generalisation fails

Source: `/data/research/benchmarks/night8/organism_v2/summary_tables.md` "Training" and "TOTAL accuracy by magnitude"; V2_REPORT §2–3. Same architecture, corpus size and schedule; the canonical target is *computed* (TOTAL = count1 + count2 as digit tokens, agents sorted alphabetically, `together` iff the two agents are equal).

| split | new relative to training | sem d_z=64 | sem d_z=256 | recon twin d_z=64 |
|---|---|---|---|---|
| in-distribution | fresh sample | 1.000 | 1.000 | 0.9997 |
| decorrelated | location class uniform | 1.000 | 1.000 | 1.000 |
| held-out agent pairs | unordered name pairs with md5 % 20 == 0 (incl. self-pairs) never in training | 0.683 | 0.685 | 0.995 |
| — distinct pairs only (n from 1,500) | | 0.987 | 0.991 | — |
| — held-out self-pairs (`A together`) | | 0.000 | 0.000 | — |
| held-out totals 80–98 | each count ≤ 49 seen, the sum never | **0.000** | **0.000** | 1.000 |
| — ones digit of the sum correct | | 0.454 | 0.911 | — |
| — decoded tens digit = 7 (training maximum band) | | 0.993 | 0.993 | — |

The same substrate family therefore shows both faces: v1 generalises across every combination it was tested on; v2 generalises alphabetical ordering to never-seen pairs (0.99) and the carry rule of addition (ones digit 0.91 at d_z=256) but memorises the *magnitude* support (tens digit clamps at the training maximum, 0.993 of held-out totals decode as 7x) and does not generalise the equality test to held-out self-pairs (0.000). The recon twin, trained on the same data to copy the surface, has no such failures (0.995 / 1.000) because its target requires no computation. Generalisation here is a property of the *target function*, not of the encoder or the corpus.

## G.7 Figure A1 — training curves [existing result]

![Figure A1: organism_v1 training curves](figures/figA1-organism-training.svg)

Rendered by `make_organism_figure.py` from the five `metrics.json` files (loss logged every 100 steps to four decimals, shown as a 2,000-step trailing mean on a log axis; exact match at the 2,000-step evaluations, first 40,000 steps shown; after step 40,000 every evaluation on all three splits is 1.000 for all five runs). Plotted values are frozen in `figures/organisms/figA1_data.json`. Loss values below 1e-4 are floor-rounded in the logs. No per-step validation loss was logged, and no curves exist for the M1/M2 battery quantities.

## G.8 What this means for the paper

These organisms are converged solutions of a small closed problem: 7.7 M parameters, 953 vocabulary types, about 4.7 distinct training tokens per parameter seen 6–15 times, and a target with an exact parser. They reach 1.000 on every behavioural measure the archive defines, including the two held-out splits, and B2 finds decoder certification at 0.99–1.00 and READ at ceiling on agent-disjoint items with seed spread at the third decimal except for the 98-way count read. Their value to TAE-Bench is exactly that: a substrate on which the ceiling is known to be 1.0, the factor labels are exact, and an untrained copy certifies 0 items, so any shortfall in a READ or EDIT cell is attributable to the interpretation method rather than to the substrate. The limits are equally definite. Generalisation was only ever tested inside the generator's support (novel factor combinations, never novel words, families or lengths); the one out-of-support datum (0.96–0.98 on verb-class × patient-type combinations the DGP never emits) was incidental; in-distribution evaluation items were not checked for string collisions with the training set; and v2 shows that the same recipe memorises rather than generalises as soon as the target requires magnitude extrapolation or an equality test on held-out pairs. Nothing on this page licenses a claim about SONAR-scale encoders or natural text. [needs to be improved] Before submission, add a train/eval string-collision count for the in-distribution split and a held-out-*name* split for v1 (the B2 stimulus split holds names out of the probe's training set, not of the organism's).

## G.9 Files touched by this page

- Source of truth: `/data/research/tae-interp/neuralese-paper/APPENDIX_ORGANISMS.md` → `appendix_organisms.html` (built by `build.py`).
- Computation: `corpus_token_stats.py` → `figures/organisms/corpus_token_stats.json` and `.log`.
- Figure: `make_organism_figure.py` → `figures/figA1-organism-training.{svg,png}`, data `figures/organisms/figA1_data.json`.
- Copied inputs: `figures/organisms/seed{1,2,3}_metrics.json`, `figures/organisms/SEEDS_REPORT.md` (from 4l; also at `/data/research/benchmarks/night8/organism_v1_seeds/SEEDS_REPORT.md`).
- Publication: the generated page and shared stylesheet are mirrored to `/data/public/safety/research/tae-interp/neuralese-paper/`.