Field notes / Experiment guide
Which letters of the opening carry the kappa signal?
The first eight letters of a rule predict held-out kappa. Is the information in a few early letters, or spread across them?
New to these ants? Start here.
An ant moves on a grid of coloured squares. Each colour tells it to turn left (L) or right (R). It advances that square to the next colour, then steps forward. A rule such as RLR is the list of turns, starting at colour zero. On a blank grid the same rule always makes the same trajectory.
Territory means every cell visited so far. Its boundary consists of visited cells with an unvisited neighbour. A highway is a repeating pattern of motion that carries the ant away. A tether means repeated returns to a small home region.
Growing like the cube root of time means taking eight times as many steps to double a linear size. The rotor walker is a comparison model that follows and rotates arrows stored on cells. A seed makes a random starting pattern reproducible; the ant itself can still move deterministically.
We distinguish c, the chance of discovering a cell per boundary step, from kappa (κ), time at the boundary normalized by perimeter divided by area. Explaining their values is harder than using them to account for growth.
The idea under test
Registered: position 1 carries nothing, positions 2-4 each carry at least 0.05 of R^2, and positions 5-8 carry no more than the largest of positions 2-4.
What we learned
Three of four predictions held and one failed: position 5 carries the most (loss 0.40), positions 2-7 each cost 0.2-0.4, and position 8 costs 0.07.
What this leaves open
The lookup is a conjunctive key, so a large loss at a position means it matters jointly with the others; main effects and interactions are not separated.
Read the evidence and reproduction recipe ↗
Inside the experiment
Methods, registered predictions, exact commands and saved analysis. These are the working research documents; expand them for the detail behind the introduction.
Methods and original experiment record
Which prefix positions carry kappa? Per-position ablation (2026-10-20)
Question
../2026-10-20-kappa-prefix-runs/ showed the prefix signal is distributed over the opening (first four runs 0.48, eight letters 0.62).
Which of letters 1-8 carry it?
Design (registered before running, 2026-10-20 05:20 +13:00, git f40da67)
- Same train / test as the sibling experiments: 2185 stationary blob census rules, 162 fresh stationary rules, disjoint; strict held-out R^2 against the training mean; lookup P8 (first 8 letters, fall back to shorter keys with >= 3 training rules).
- Ablation of position j (j = 1..8): the key ignores letter j (replaced by
*in the key for all depths that include it). - Group-count control: ablating a position merges groups, which changes the bias-variance trade-off independently of information. Control: permute the letters at position j among training rules (200 permutations, seed 20261020) and re-run the lookup unchanged on the test rules; this keeps the number of groups and destroys the information at j. Report the mean control R^2.
- Loss = R^2(P8) - R^2(ablated). Bootstrap over test rules (2000 resamples) for the ablation loss.
Predictions (written before the run)
- A1: position 1 (always R in this cohort) has loss within 0.01 of 0.
- A2: each of positions 2, 3, 4 has ablation loss >= 0.05.
- A3: no position in 5-8 has a larger ablation loss than the largest loss among positions 2-4 (information is front-loaded).
- A4: for each position the permutation-control loss is at least as large as the masking loss minus 0.05 (masking is not hiding a large information effect behind regrouping; failure means the masking and the control disagree).
Failures are reported as failures; anything chosen after seeing the results is labelled post hoc.
Results (results/ablate.txt; 2185 train, 162 test, 0 in both; P8 R^2 +0.624)
| position | R^2 with letter ablated | loss [95% CI] | permutation control loss |
|---|---|---|---|
| 1 | +0.624 | +0.000 | +0.000 |
| 2 | +0.395 | +0.229 [+0.128, +0.331] | +0.252 |
| 3 | +0.370 | +0.255 [+0.138, +0.383] | +0.280 |
| 4 | +0.279 | +0.345 [+0.223, +0.474] | +0.388 |
| 5 | +0.224 | +0.400 [+0.230, +0.565] | +0.442 |
| 6 | +0.330 | +0.294 [+0.171, +0.405] | +0.321 |
| 7 | +0.408 | +0.216 [+0.096, +0.321] | +0.234 |
| 8 | +0.552 | +0.073 [-0.002, +0.157] | +0.099 |
A1 HELD, A2 HELD, A3 FAILED, A4 HELD.
Reading. The information is not front-loaded: ablating position 5 costs the most (0.40), and positions 2-7 each cost 0.2-0.4, whereas position 8 costs 0.07 (interval includes 0). The permutation control, which keeps the group count and destroys only the information at the position, gives slightly larger losses at every position, so the masking result is not a regrouping artefact. Caveat that limits the reading: the lookup is a conjunctive key over the whole prefix, so a large loss at one position says that position matters in combination with the others, not that it matters additively; it cannot separate main effects from interactions, and the lost R^2 across positions sums to far more than 0.62. Which letter patterns matter (for instance, where the first L-to-R changes fall) is untested. Together with the run-structure result (lead alone 0, four runs 0.48) this says the signal is a property of roughly letters 2-7 taken together.
Reproduce
uv run experiments/2026-10-20-kappa-prefix-ablation/scripts/ablate.py -> results/ablate.txt, results/ablate.json.
Analysis notebook: notebook.ipynb
1 saved outputs · 1 code cells. Cells are not executed during publication. Download notebook
Which prefix positions carry kappa?
Registered ablation (A1-A4 committed before the run; see README). Same split as the sibling experiments: 2185 stationary census rules train, 162 fresh stationary rules test, disjoint. The 8-letter lookup P8 is re-scored with the key ignoring letter j (masking), and, as a group-count control, with the letters at position j permuted among training rules (200 permutations).
Assumption. Same stationarity filter and fresh-rule target as the W3c audit.
Code
import json
from pathlib import Path
root = next(p for p in [Path.cwd(), *Path.cwd().parents] if (p / "AGENTS.md").exists())
d = json.loads((root / "experiments/2026-10-20-kappa-prefix-ablation/results/ablate.json").read_text())
print("inputs:", {k: v[:12] for k, v in d["input_sha256"].items()}, "train", d["n_train"], "test", d["n_test"], "overlap", d["overlap"])
print(f"P8 R^2 {d['r2_base']:+.3f}")
for j, r in d["positions"].items():
lo, hi = r["loss_ci"]
print(f"pos {j}: loss {r['loss']:+.3f} [{lo:+.3f}, {hi:+.3f}] control loss {r['control_loss']:+.3f}")
print(d["verdicts"])
assert d["overlap"] == 0
assert [v for v in d["verdicts"].values()] == [True, True, False, True] # A3 failed, as in the README
loss = {int(j): r["loss"] for j, r in d["positions"].items()}
assert max(loss, key=loss.get) == 5 and loss[8] < 0.1
What this shows and does not show
- Information is not front-loaded (A3 failed): ablating position 5 costs the most (0.40), positions 2-7 each cost 0.2-0.4, position 8 costs 0.07 with an interval that includes 0. Position 1 is constant in this cohort and costs nothing.
- The permutation control agrees with masking at every position, so this is not a regrouping artifact.
- Not shown: whether the signal is additive over positions or needs interactions. The lookup is a conjunctive key, and the losses sum to far more than the 0.62 total, which points to interaction or redundancy but does not separate them.
Next
A registered additive (letter-by-position) model against the full 8-letter lookup on the same split.
Read the dated lab log
Lab notebook: kappa prefix ablation
Append-only. Newest entry last. Never edit an old entry; correct it with a new one
(corrects: <timestamp>). Tags: intent, assumption, observation, interpretation,
surprise, decision, dead-end, question, deviation. See experiments/README.md.
2026-10-20 05:20 (+13:00) · f40da67 · intent
Find which of letters 1-8 carry the prefix signal for ln kappa. Predictions A1-A4 registered in README and committed (593d127) before scripts/ablate.py existed. A permutation control keeps the group count fixed so masking is not confused with regrouping.
2026-10-20 05:30 (+13:00) · 593d127 · observation
Ran ablate.py (seed 20261020, 2000 bootstraps, 200 permutations). Loss in held-out R^2 from ablating position 1..8: 0.00, 0.23, 0.26, 0.35, 0.40, 0.29, 0.22, 0.07 (P8 R^2 0.624). A1, A2, A4 held; A3 FAILED.
2026-10-20 05:35 (+13:00) · 593d127 · surprise
I expected the information to be front-loaded (the lead-run picture). Position 5 costs the most and positions 6-7 cost as much as positions 2-3. Only position 8 is nearly free.
2026-10-20 05:40 (+13:00) · 593d127 · interpretation
The signal is spread over roughly letters 2-7. The lookup is a conjunctive key, so a large loss at one position means it matters jointly with the others; the losses sum to far more than the 0.62 total, which is the signature of interaction or redundancy. This experiment cannot separate main effects from interactions. The permutation control agrees with masking at every position (A4).
2026-10-20 05:40 (+13:00) · 593d127 · question
Does an additive model (letter-by-position effects, ridge) reach 0.6, or does the signal need interactions? A registered comparison of an additive model with the full 8-letter lookup on the same split would answer it.