Developmental checkpoints. Plan

Status 2026-10-30: analysis tooling written and self-tested; GPU work waits for a rented box. Context: ../../research-directions.md section 2a; ../../PREMISE.md.

Goal

Take each consciousness-relevant measure that has a published protocol and trace it over training on open checkpoints. Each trajectory comes with an untrained null, a comparison against the loss curve, and the data seen before any onset. The deliverable is the checkpoint atlas.

Model families

• Pythia (Biderman et al. 2023), 70M to 12B. Each model has 154 checkpoints (steps 0, 1, 2, 4, ..., 512, then every 1,000 to 143,000). It was trained on the Pile in a reproducible batch order, so we can list the data seen before step t exactly. There are deduplicated and non-deduplicated variants. Step 0 is the random-init null.
• OLMo 2 (Ai2, 2024-2025), 7B, 13B and 32B. Intermediate pretraining checkpoints are released, the data (Dolma, Dolmino) is public, and the post-training stages (SFT, DPO, RLVR) come out as separate models. We use it for measures that need instruction following.
• Before use, check whether OLMo 3 or a newer Pythia-style release exists with the same guarantees, and record the answer in reading.md.

Measures, in order of cost

M1. Valence axis. Contrastive prompt pairs (positive and negative affect words in neutral frames) give a direction per layer. We test its causal efficacy by steering along it and scoring continuation sentiment with a fixed classifier. Per step we report whether the direction exists (probe accuracy against a shuffled-label null) and the steering effect size.

M2. Workspace structure. We reimplement the Jacobian-lens method of Gurnee, Sofroniew and Lindsey (July 2026). A per-layer mean Jacobian of the final residual stream maps each layer's activations to the vocabulary, and the "J-space" is sparse nonnegative combinations of the resulting per-token vectors (corrected 2026-10-30; see ../02-open-replication/method-r1.md). We score its dimensionality and the five properties the authors list (verbal report, directed modulation, internal reasoning, flexible generalisation, selectivity), each per step. The behavioural properties need prompt following, so on Pythia only the structural signatures (occupancy, variance explained, broadcast) are scored at every step. The tooling is shared with ../02-open-replication/.

M3. Self-versus-other probe. The checkpoint and a reference model generate continuations on the same prompts, and a linear probe on residual-stream activations learns to classify the source. We report probe accuracy per step against a shuffled null. Onset is the first step where accuracy beats the null by a pre-registered margin.

M5. Training-signal residual, which is what survives of Tree A's ledger. It is the behaviour bits on a fixed family of probe tasks, minus the training-signal bits consumed up to that checkpoint, with the step-0 competency subtracted. It tracks sample efficiency over training. Definitions are in ../../trees/01-a-ledger/writeup.md section 4.

M4 (OLMo stages only). Concept-injection introspection (Lindsey 2025, with the content-agnostic replication by Lederman and Mahowald 2026) and self-report under self-referential prompting (Berg et al. 2025), measured on base, SFT, DPO and RLVR.

Design

• Sizes: Pythia 160M, 410M, 1.4B and 2.8B. The 2.8B model in fp16 fits on a single 24 GB GPU.
• Checkpoints: steps 0, 1, 2, 4, 8, ..., 512, 1000, 2000, 4000, 8000, 16000, 32000, 64000, 100000, 143000 (about 20).
• Nulls: step 0, shuffled-label probes, random-direction steering.
• Pre-registered analysis. For each measure and size we ask (a) whether it is above null at step 0, (b) whether it correlates with loss beyond what step explains, and (c) whether there is a change-point, meaning a step where the slope changes by more than a stated threshold. If there is one, we give the batch range before it.
• Data provenance. For any change-point we retrieve the Pile batches in the preceding interval and report their composition by source.

Steps

1. Tooling: checkpoint loader, activation capture, probe trainer, steering harness. Stdlib plus torch and the model library, pinned in a uv lockfile, with fixed seeds and no network in tests.
2. Run M1 and M3 end to end on Pythia 160M, all checkpoints, on a rented GPU, and fix the pipeline before scaling.
3. Pre-register the analysis plan on OSF.
4. M1, M2 and M3 on all four sizes.
5. Atlas write-up, with figures per measure, the change-point table, and the data provenance for each onset.
6. OLMo 2 stages with M4.

Decision gates

• After step 2 the pipeline must reproduce a known result: probe accuracy on a trained checkpoint well above step 0. If step 0 already scores high on M3, we have a finding, and the atlas goes ahead with it as the headline.
• After step 5 we publish whatever shape the results take. A flat, a smooth or a stepwise outcome each tells us something (see summary.md).

Cost

About 30 to 60 rented GPU hours for the Pythia atlas. Checkpoints need disk: 2.8B x 20 checkpoints at fp16 is about 110 GB, streamed and deleted and never stored on this box. The only work done here is analysis of exported tables.

Relation to the other lines

Once the ledger (../../trees/01-a-ledger/) exists, it supplies the residual as a function of training step. ../02-open-replication/ shares the M2 tooling, and ../03-adversarial-registry/ supplies the pre-registration template.