Adversarial registry: protocol

Draft 2, 2026-10-30; draft 1 2026-09-30. Adapted from the Templeton World Charity Foundation's structured adversarial collaborations and the Cogitate consortium's published method, cut down to what a small executor can run on open models. Reviewed against Tree G's checklist (../../trees/02-g-theory/checklist.md) on 2026-10-30; section 4 gained four rules from it.

1. What can be registered

A claim about a machine-mind property qualifies when all four of these hold.

• It concerns a measurement on an open-weight model or an untrained system, and anyone with the listed hardware can reproduce it.
• At least two camps predict different outcomes for the same measurement, each with a numeric threshold.
• An executor with no stake in either camp can run the measurement.
• No prompt in the protocol contains the hypothesis, mind vocabulary, or any instruction that would tell the model what is being tested.

Some claims depend on closed models, or on self-report as the only evidence, or on a theory that yields no divergent measurable prediction. IIT's substrate claim is the standing example of the last. Such claims are listed as open problems and do not become entries.

2. Roles

• Proposer camps, two or more. Each supplies a prediction, a threshold, and the interpretation it commits to for each outcome.
• Executor. Runs the measurement exactly as frozen and has no prior published position on the claim. This can be this institution, Eleos, or a named third party.
• Reviewer. One person who expects a null result, named before the freeze, who reviews the pre-registration and the analysis code.
• Host. Publishes the entry and the result, and keeps the freeze date and the unedited pre-registration.

3. Sequence

1. Proposal. A camp or the host drafts an entry from entry-template.md.
2. Negotiation. The camps agree on models, versions, sample sizes, nulls (untrained model, random directions, shuffled labels), the metric and the thresholds. A camp that declines to name a threshold accepts the executor's, and the entry records that it did. A camp that declines both is listed as "did not commit", and the entry goes ahead without it as long as another camp remains.
3. Freeze. The entry is posted with a date and a hash. After that nothing above the result section changes. Corrections are appended and dated.
4. Execution. The executor runs the frozen protocol and records seeds, model hashes, code commit, hardware and runtime. Until the pre-registered analysis is complete, the executor does not look at analyses that favour either camp.
5. Result. The host posts the full table, the pre-registered verdict against each threshold, and each camp's pre-committed interpretation, verbatim. Exploratory analyses follow, labelled as exploratory.
6. Reply. Within 30 days each camp may append a dated response, which the host adds unedited.

4. Rules for the measurement

• Open weights only, with model version and hash recorded.
• An untrained control of the same architecture wherever the claim allows one.
• A metric-substitution control: alongside the primary metric, at least one alternative that the camps agree is a fair reading.
• Seeds are fixed and listed. Sample sizes come from a power estimate stated in the entry.
• The code is public before execution.
• The entry names the task family the claim covers and how performance on it is counted (checklist row 1).
• Implementation choices the claim should not depend on (prompt wording, layer, seed, decoding settings) are varied, and their effect is reported beside the claimed one (row 3).
• Besides chance and the nulls, the result is compared with the simplest known mechanism that could produce it at the same cost (row 4).
• Means come with medians, per-run spread, and the number of runs that carry the effect (row 8).
• Every entry answers all thirteen checklist rows by number. A row it cannot answer is recorded as unanswered.

5. What the registry publishes

Each entry has its frozen pre-registration, the result, the interpretations, the replies, and a one-line status (proposed, negotiating, frozen, executed, replied, closed). Each open problem has the claim, the reason it cannot be entered, and what would change that.

6. Disputes

For the purposes of the entry, the executor's reading of the frozen protocol stands. A camp that believes the protocol was not followed says so in its reply and names the step. The host then asks the reviewer to check and appends the finding. No entry is withdrawn after the freeze.