spine/01-developmental-checkpoints · analysis tooling written and self-tested; GPU work waits for a

Developmental checkpoints. Summary in plain language

The question

Every published measurement of a "consciousness indicator" in a language model was taken on a finished model. None of those papers asks when, during training, the measured thing appeared, what the model had read just before, or whether it came in gradually or all at once. In July 2026 the field described this kind of developmental evidence as "an important but underdeveloped source of insight". It is also the question behind Levin's "what did selection pay for" logic, applied to a model in place of an organism.

Two model families let us answer it without training anything. Pythia (EleutherAI) publishes 154 checkpoints per model, from step 0 to the end of training, for eight sizes between 70M and 12B parameters. It also publishes the training data in the exact order the model saw it. OLMo 2 (Ai2) publishes intermediate checkpoints and the data, and releases each post-training stage (supervised fine-tuning, preference tuning, reinforcement learning) as its own model. Step 0 of Pythia is an untrained network, so we get the random-substrate null for free.

Where to start

We begin with three measures that already have published protocols and need only inference.

A valence axis. At each checkpoint, build the "emotion vector" direction (Anthropic, April 2026) and test whether steering along it changes behaviour. When does a valence-like direction first exist, and when does it first do anything?
Workspace structure. The Jacobian-lens method (Anthropic, July 2026) picks out the small set of representations a model is poised to verbalise. We run it at each checkpoint and score its five workspace properties, to see whether the structure grows smoothly with loss or appears in a step.
A self-versus-other signal. At each checkpoint we train a linear probe to tell the model's own continuations from another model's. It is the cheapest precursor of a self-model, and the one most likely to show a developmental onset.

We run all three on Pythia 160M, 410M, 1.4B and 2.8B, at about twenty checkpoints each, and plot them against training step and against the loss curve. Where a measure switches on, we list the data batches the model saw in the interval before. Measures that need instruction-following (introspection under concept injection, self-report under self-reference) come next, repeated across the OLMo 2 post-training stages to find which stage creates them.

What the outcomes could be

A measure is already present at step 0 and barely changes. Then it belongs to the architecture and owes nothing to learning, and every snapshot result that reported it in a trained model was in fact reporting the substrate. That would carry the "random networks already do it" finding to a new place, and it would matter.
A measure tracks the loss curve smoothly. There is then nothing developmental to see. The indicator comes along as the model gets better at prediction.
A measure appears at a particular step, with identifiable data before it. That would be a developmental event, the first on record, and the preceding data would give us a hypothesis about what the indicator is made of.
A post-training stage creates a measure that pretraining did not. The indicator would then be a product of the alignment recipe, as the mimicry critique predicts, and we would have a number for that prediction.

What kind of research this is

Inference on public models with public data. The compute bill is small (tens of rented GPU hours) and most of the effort goes into measuring carefully. The output is an atlas of each measure by training step and model size, with the untrained null built in.

Timeline