Field notes / Experiment guide

Does the edge model beat a simple lookup table?

A model relating turning bias to fresh-ground entry looks predictive. Does it explain more than a few early rule letters already tell us?

New to these ants? Start here.

An ant moves on a grid of coloured squares. Each colour tells it to turn left (L) or right (R). It advances that square to the next colour, then steps forward. A rule such as RLR is the list of turns, starting at colour zero. On a blank grid the same rule always makes the same trajectory.

Territory means every cell visited so far. Its boundary consists of visited cells with an unvisited neighbour. A highway is a repeating pattern of motion that carries the ant away. A tether means repeated returns to a small home region.

Growing like the cube root of time means taking eight times as many steps to double a linear size. The rotor walker is a comparison model that follows and rotates arrows stored on cells. A seed makes a random starting pattern reproducible; the ant itself can still move deterministically.

We distinguish c, the chance of discovering a cell per boundary step, from kappa (κ), time at the boundary normalized by perimeter divided by area. Explaining their values is harder than using them to account for growth.

The idea under test

A useful mechanistic predictor should outperform coarse letter-group baselines on rules held out of training.

What we learned

The chirality model beats one-letter grouping, but a three-letter lookup predicts more accurately. Its apparent success does not isolate a mechanism.

What this leaves open

The original quoted score confused correlation with predictive R-squared. Bootstrap intervals condition on the fitted models and omit training and seed uncertainty.

Read the evidence and reproduction recipe ↗

Inside the experiment

Methods, registered predictions, exact commands and saved analysis. These are the working research documents; expand them for the detail behind the introduction.

Methods and original experiment record

Does chirality predict more than letter groups?

Registered 2026-10-15 23:17 NZDT, before scoring

Base commit: b3a0f762ac2230b5df92188206a4d3a1081897be, with existing uncommitted work retained. This is a critique-driven reanalysis of an already inspected scatter plot, not an independent prospective validation.

Question: how much does the existing chirality model add beyond coarse groups defined by rule letters, on exactly its original held-out rules?

Prediction: the primary one-letter baseline explains most of the pooled variation. Operationally, its held-out R² will be within 0.05 of chirality. We also expect chirality to reduce squared error by less than 10% against the three-letter baseline. Both predictions can fail; report all scores.

Fixed method, without selecting groups using measured or predicted c:

Read 2026-10-13-probe-growth-census/results/c_model.csv. Retain its non-bursty cohort, targets, predictions and held_out flag exactly.
Primary baseline: calibration-target mean by rule[1] (the letter after the leading R). Secondary baseline: mean by rule[1:4], falling back to the primary group mean, then the overall calibration mean, for absent groups. Short rules have their available suffix as a separate group.
Fit means only on eligible non-held-out rows. Original chirality constants used the full 300-rule edge-statistics sample; report calibration-rule overlap and eligible counts explicitly, rather than claiming equal targets.
Report original R² (held-out-mean denominator), strict R² (calibration-mean predictor denominator), SSE, RMSE, MAE, and paired SSE reduction of chirality versus each baseline, globally and within each predefined letter group. A within-group R² uses that held-out group's mean; it is a descriptive denominator, never a fitted prediction. Also report pooled within-group R² after centering both observations and predictions within held-out groups: this isolates discrimination from group-mean calibration error.
Paired rule bootstrap: 4000 resamples, seed 20261046, stratified by the primary letter group, with fitted models fixed. Report percentile 95% intervals for SSE reduction and delta R². These quantify held-out rule variation conditional on the fitted models, not seed noise or training uncertainty. No ant trajectories or probe estimators change.
Synthetic controls verify zero residual skill for group-constant predictions, perfect residual skill for a correct predictor, negative skill for a wrong predictor, and training-only means/fallbacks. Do not infer within-group prediction from the extremes in a scatter plot alone.

Inputs are committed source results; outputs belong only in this folder. Save per-rule predictions, group statistics, a machine-readable summary with input/script hashes and exact command, and an executed shareable notebook.

Reproduction

From WSL at /mnt/f/Code/turmite, first set:

export UV_PROJECT_ENVIRONMENT=$HOME/.venvs/turmite UV_LINK_MODE=copy
uv run experiments/2026-10-15-chirality-baselines/scripts/compare.py
uv run experiments/2026-10-15-chirality-baselines/scripts/check_controls.py
uv run --with nbconvert --with ipykernel --with nbformat jupyter nbconvert --to notebook --execute --inplace experiments/2026-10-15-chirality-baselines/notebook.ipynb
cargo test --release

Results (2026-10-15 23:23 NZDT)

The original cohort and predictions reconstruct exactly (tolerance 1e-14). All 300 calibration rules have eligible targets; 2,747 rules are held out. Results are in results/comparison.json, per-group groups.{csv,json}, and per-rule held_out_predictions.csv; source hashes and command are embedded.

Prediction	Held-out R²	RMSE
One-letter mean	0.585589	0.081457
Three-letter mean	0.807648	0.055496
Chirality	0.660023	0.073780

Chirality reduces SSE by 17.96% [15.92, 20.08]% against the one-letter mean, but has 76.75% [65.42, 88.93]% more SSE than the three-letter mean. Intervals are the registered conditional paired rule bootstrap. Three-letter fallback is needed for the short held-out rule RLL; no target-dependent selection.

Within the primary L/R groups (1,452/1,295 test rules; 173/127 training rules), chirality R² is 0.189960/0.163770. Its pooled centered within-three-letter R² is -0.00006245; the raw within-group R², retaining group-mean bias, is -0.775530. Centered scores are descriptive, not predictions fitted on test data. The first registered prediction fails (one-letter gap 0.07443 > 0.05). The second holds (chirality underperforms the three-letter baseline).

The critique's R²=0.88 is not reproduced: the original files already record R²=0.660 and correlation 0.869. Chirality has modest information within broad groups; after conditioning on three letters there is essentially no residual discrimination. These data do not establish a growth mechanism or seed robustness. Rule-bootstrap intervals condition on fitted models and assume independent rules despite related/repeated rule strings. No physical instrument or window changed, so walker calibration was not rerun. Synthetic controls pass.

The executed notebook.ipynb reads the small versionable result files only; new files await a user-requested commit. No dependencies were added. The paper extracts results with uv run research-outputs/cube-root-growth-paper/chirality_baseline_numbers.py.

Target definition: the existing probe_summary.csv uses final-window new_cells / boundary_steps at k=26 (the octave from 2^25 to 2^26), as defined in experiments/_lib/analyse_probe.py; exponent fits use k=20..26 but c here is the final-window value, not a fitted or pooled c. This reanalysis inherits the saved calibration membership and blank-board targets; it draws no new train/test split or initial conditions.

Verification: three synthetic control tests pass; cargo test --release passes; the notebook executes three code cells cleanly; repeating all four analysis outputs gives identical bytes. The paper compiles without diagnostics; pages 1, 2, 20 and 21 were rendered and reviewed. Verification hashes and exact command are in results/verification.json. To repeat:

uv run --with pymupdf experiments/2026-10-15-chirality-baselines/scripts/verify_artifacts.py

PyMuPDF is a one-off verification dependency under uv, not a change to the project dependency declarations. Rendered inspection pages stay in ignored scratch/. The PDF workflow used bundled Node on Windows because WSL has no node, and PyMuPDF because Poppler's pdftotext is unavailable in WSL.

Analysis notebook: notebook.ipynb

3 saved outputs · 3 code cells. Cells are not executed during publication. Download notebook

Question and why it matters

Does the chirality model predict edge-entry probability beyond coarse letter groups? A high pooled score need not imply within-group prediction. See registration and recipe and scientific record.

Assumptions and definitions

Verified: the saved cohort, target values, calibration membership and chirality predictions reconstruct exactly (tolerance 1e-14). Fit group means on the same 300 calibration rules; score all 2,747 original held-out rules. Primary groups use rule[1]; secondary groups use rule[1:4], with training-only fallback. No grouping uses measured c. This is a reanalysis after inspection of the original plot.

Code

In [1]:

from pathlib import Path
import hashlib, json
import numpy as np
import polars as pl
SEED = 20261046
ROOT = next(p for p in [Path.cwd(), *Path.cwd().parents] if (p / "Cargo.toml").exists())
E = ROOT / "experiments/2026-10-15-chirality-baselines/results"
paths = [E / "comparison.json", E / "groups.csv", E / "held_out_predictions.csv"]
assert all(p.is_file() for p in paths)
s = json.loads(paths[0].read_text())
assert s["seed"] == SEED
print("Registration date: 2026-10-15 NZDT; seed:", SEED)
print(json.dumps(s["provenance"], indent=2))
for p in paths:
    print(p.relative_to(ROOT), hashlib.sha256(p.read_bytes()).hexdigest())

Registration date: 2026-10-15 NZDT; seed: 20261046
{
  "git_commit": "b3a0f762ac2230b5df92188206a4d3a1081897be",
  "working_tree": "Uncommitted changes present; exact input/script hashes below",
  "command": "uv run experiments/2026-10-15-chirality-baselines/scripts/compare.py",
  "python": "3.12.3 (main, Aug 31 2026, 10:18:26) [GCC 13.3.0]",
  "numpy": "2.5.4",
  "polars": "2.0.0",
  "inputs_sha256": {
    "experiments/2026-10-13-probe-growth-census/results/c_model.csv": "5378ca19616dd6e4c5f894403cd74afa72e7d3092d18d600031b58150779b42e",
    "experiments/2026-10-13-probe-growth-census/results/probe_summary.csv": "c76c7927185b1cee273433c340cfa393fb16696fbe4a6f155fc49c33ef32cb5d",
    "experiments/2026-10-13-edge-statistics/results/edgestats_sample.csv": "976b1359224fdbf511ab1ac4d876854da0c9acf83e9eff5a39180e3845231464",
    "experiments/2026-10-13-probe-growth-census/scripts/c_model.py": "c754e2d7854ff9be115e9c89c1e1fbc8203ff1b9c42b40ca82e5b2ba040393cf",
    "uv.lock": "598ec0fc1978845ded5016368a2b1f2807de61437a97274fbd8c5a8cd746412b",
    "experiments/2026-10-15-chirality-baselines/scripts/compare.py": "6757570c57e842f5387ec9bb2e0a1b1e6cb81790ea3f88a7f65716c3b98e9889"
  }
}
experiments/2026-10-15-chirality-baselines/results/comparison.json 1a73b6782ebb8a78b72b8c74e39538fe8c45b22a0f9171984b9a57917388e5d4
experiments/2026-10-15-chirality-baselines/results/groups.csv ad52a0920af7f84cf7e986fb4327394cc580d87c663b7bb3bc5effb29f6f89be
experiments/2026-10-15-chirality-baselines/results/held_out_predictions.csv 1edf93da38169a4ad3fb12373b28272c43751150d2cd460490eacdd2660e271b

Data and method

The producing script is scripts/compare.py; it writes per-rule predictions and subgroup statistics. Conventional R² uses the held-out mean only as a scoring denominator; strict R² compares with the training-mean predictor. RMSE and SSE compare actual predictions. Centered within-group R² subtracts each held-out group's mean from both observations and predictions, isolating discrimination from group-mean bias; these centering operations do not define a deployable predictor. Bootstrap intervals resample paired held-out rules within the two primary groups, keeping model fits fixed (4,000 draws).

Results

Code

In [2]:

table = pl.DataFrame([{"model": k, **{m: v[m] for m in ["n", "r2", "strict_r2", "sse", "rmse", "mae"]}} for k, v in s["overall"].items()])
print(table)
print(json.dumps(s["comparisons"], indent=2))
assert abs(s["overall"]["chirality"]["r2"] - 0.6600227968106389) < 1e-12

shape: (4, 7)
┌────────────┬──────┬───────────┬───────────┬───────────┬──────────┬──────────┐
│ model      ┆ n    ┆ r2        ┆ strict_r2 ┆ sse       ┆ rmse     ┆ mae      │
│ ---        ┆ ---  ┆ ---       ┆ ---       ┆ ---       ┆ ---      ┆ ---      │
│ str        ┆ i64  ┆ f64       ┆ f64       ┆ f64       ┆ f64      ┆ f64      │
╞════════════╪══════╪═══════════╪═══════════╪═══════════╪══════════╪══════════╡
│ constant   ┆ 2747 ┆ -0.004051 ┆ 0.0       ┆ 44.160921 ┆ 0.126791 ┆ 0.099676 │
│ letter1    ┆ 2747 ┆ 0.585589  ┆ 0.587261  ┆ 18.226949 ┆ 0.081457 ┆ 0.066907 │
│ letters123 ┆ 2747 ┆ 0.807648  ┆ 0.808424  ┆ 8.460185  ┆ 0.055496 ┆ 0.037915 │
│ chirality  ┆ 2747 ┆ 0.660023  ┆ 0.661395  ┆ 14.953126 ┆ 0.07378  ┆ 0.05574  │
└────────────┴──────┴───────────┴───────────┴───────────┴──────────┴──────────┘
{
  "letter1": {
    "sse_reduction": 0.17961444222175516,
    "sse_reduction_ci": [
      0.15916923770539063,
      0.20078881594334483
    ],
    "delta_r2": 0.07443428902422923,
    "delta_r2_ci": [
      0.0647563625009031,
      0.0849674095077579
    ]
  },
  "letters123": {
    "sse_reduction": -0.7674703172548021,
    "sse_reduction_ci": [
      -0.8893410252511816,
      -0.6541756975241081
    ],
    "delta_r2": -0.1476247773124691,
    "delta_r2_ci": [
      -0.15786211447615256,
      -0.13674597543355496
    ]
  }
}

Chirality improves on one-letter grouping but the three-letter lookup predicts more accurately. These are held-out predictions of c, not evidence for the origin of the growth exponent. The quoted R²=0.88 is not reproduced: existing results give R²=0.660 and correlation 0.869.

Code

In [3]:

groups = pl.read_csv(E / "groups.csv")
print(groups.filter((pl.col("grouping") == "letter1") & (pl.col("model") != "constant")).select("group", "model", "n", "n_train_group", "r2", "rmse", "vs_baseline_sse_reduction"))
print(json.dumps(s["grouping"], indent=2))
print("Registered prediction outcomes:", s["registered_predictions"])

shape: (6, 7)
┌───────┬────────────┬──────┬───────────────┬───────────┬──────────┬───────────────────────────┐
│ group ┆ model      ┆ n    ┆ n_train_group ┆ r2        ┆ rmse     ┆ vs_baseline_sse_reduction │
│ ---   ┆ ---        ┆ ---  ┆ ---           ┆ ---       ┆ ---      ┆ ---                       │
│ str   ┆ str        ┆ i64  ┆ i64           ┆ f64       ┆ f64      ┆ f64                       │
╞═══════╪════════════╪══════╪═══════════════╪═══════════╪══════════╪═══════════════════════════╡
│ L     ┆ letter1    ┆ 1452 ┆ 173           ┆ -0.001194 ┆ 0.085419 ┆ 0.0                       │
│ L     ┆ letters123 ┆ 1452 ┆ 173           ┆ 0.369208  ┆ 0.067802 ┆ 0.369961                  │
│ L     ┆ chirality  ┆ 1452 ┆ 173           ┆ 0.18996   ┆ 0.076834 ┆ 0.190927                  │
│ R     ┆ letter1    ┆ 1295 ┆ 127           ┆ -0.000171 ┆ 0.076771 ┆ 0.0                       │
│ R     ┆ letters123 ┆ 1295 ┆ 127           ┆ 0.766059  ┆ 0.037129 ┆ 0.766099                  │
│ R     ┆ chirality  ┆ 1295 ┆ 127           ┆ 0.16377   ┆ 0.070198 ┆ 0.163912                  │
└───────┴────────────┴──────┴───────────────┴───────────┴──────────┴───────────────────────────┘
{
  "letter1": {
    "between_group_variance_fraction": 0.5859054548438272,
    "centered_within_r2": {
      "constant": -2.220446049250313e-16,
      "letter1": -2.220446049250313e-16,
      "letters123": 0.5367766803100051,
      "chirality": 0.31388122138396224
    },
    "raw_within_r2": {
      "constant": -1.4246911050549196,
      "letter1": -0.0007653978086039626,
      "letters123": 0.5354866947007293,
      "chirality": 0.17898652091362133
    }
  },
  "letters123": {
    "between_group_variance_fraction": 0.8085207286689892,
    "centered_within_r2": {
      "constant": 0.0,
      "letter1": 2.220446049250313e-16,
      "letters123": 0.0,
      "chirality": -6.244536208699358e-05
    },
    "raw_within_r2": {
      "constant": -4.243655635999513,
      "letter1": -1.1642629478007347,
      "letters123": -0.004560047360800956,
      "chirality": -0.7755300656102941
    }
  }
}
Registered prediction outcomes: {'letter1_within_005_r2': False, 'chirality_sse_gain_vs_letters123_below_010': True}

Chirality has nonzero skill inside the broad one-letter groups (R²=0.190 and 0.164), but centered within the three-letter groups its pooled R² is approximately zero. The latter describes residual discrimination; it does not establish that every individual group has exactly zero skill. The prediction that one-letter grouping lies within 0.05 of chirality fails; the prediction of less than 10% gain over three-letter grouping holds, with chirality actually worse.

Caveats and controls

Synthetic controls test perfect, zero and adverse residual prediction, group-mean shifts, training-only fits, fallback and paired bootstrap reproducibility. They do not validate a physical mechanism. No trajectory estimator or fit window changed, so rotor/random-walk calibration was not rerun. The current cohort includes related/repeated rules; rule bootstrap assumes independent sampling units and conditions on this training sample. No between-seed uncertainty or fresh-rule confirmation is available. Sparse/short groups are retained with explicit fallback and undefined within-group R² where variance is zero.

What this changes / next steps

Present the three-letter lookup as the stronger predictive baseline and qualify the chirality model's mechanistic interpretation. The coarse one-letter description alone understates the chirality model's residual information. A future independent test should hold out rule families and recalibrate across initial seeds. No claim that c alone diagnoses cube-root growth follows from these comparisons.

Read the dated lab log

Lab notebook: chirality versus letter-group baselines

2026-10-15 23:17 (+13:00) · b3a0f76 · intent

The user requested a direct training-only baseline comparison and within-group scores in response to the paper critique. Register groups and predictions in README.md before scoring; retain the original holdout and existing model.

2026-10-15 23:17 (+13:00) · b3a0f76 · assumption

The saved c_model.csv is the original model cohort and split. Verify against probe_summary.csv and the 300-rule edgestats_sample.csv, and reconstruct the chirality predictions independently before trusting its scores. Reusing this inspected dataset limits the evidential status of this reanalysis.

2026-10-15 23:20 (+13:00) · b3a0f76 · observation

uv run scripts/check_controls.py (with the experiment path prefix) passed three synthetic tests. uv run experiments/2026-10-15-chirality-baselines/scripts/compare.py produced results/comparison.json, groups.{csv,json} and held_out_predictions.csv: all 300 calibration rules have eligible targets; 2,747 held-out rules reconstruct exactly. R²: one letter 0.58559, three letters 0.80765, chirality 0.66002. The first registered prediction fails; the second holds. Original predictions match reconstruction to 1e-14.

2026-10-15 23:20 (+13:00) · b3a0f76 · surprise

The critique's R²=0.88 is not present in the saved predictions or current paper numbers. Both existing followup_stats.json and numbers.json already give R²=0.660, correlation 0.869. The primary-agent response repeated the critique's number without verifying it. Correct the premise explicitly.

2026-10-15 23:20 (+13:00) · b3a0f76 · interpretation

Chirality is not wholly uninformative within broad letter groups: their within-group R² is 0.190 (L) and 0.164 (R). It nonetheless underperforms the three-letter lookup and has essentially no pooled centered residual skill within three-letter groups. Group-mean error and discrimination must be kept distinct. These are observational comparisons of existing targets; they do not establish the mechanism of growth.

2026-10-15 23:27 (+13:00) · b3a0f76 · observation

Regeneration by scripts/verify_artifacts.py (full uv command in README) reproduces all four result files byte-for-byte; hashes in results/verification.json. The three synthetic tests pass, the shareable notebook executed three code cells without errors, and cargo test --release passes (results/release-tests.log). The paper compiles without diagnostics; the 38-page PDF's changed opening/comparison pages (1, 2, 20, 21) were rendered for visual review. Recipe and PDF hash are in the verification record.

2026-10-15 23:27 (+13:00) · b3a0f76 · deviation

WSL has no Node or Poppler pdftotext. Used bundled Windows Node for the PDF workflow initialization and a one-off uv run --with pymupdf renderer for visual verification. No project dependencies changed. Physical calibration was not repeated because no trajectory instrument/estimator/window changed; synthetic statistical controls were run instead, as registered.

2026-10-15 23:27 (+13:00) · b3a0f76 · decision

Complete: scripts, machine-readable results, lab/shareable notebooks, scientific note and manuscript comparison agree. Updated the abstract, key-results summary, evidence table and figure caption to acknowledge the stronger three-letter baseline. Retained all pre-existing work and made no commit. Future work: independent rule-family holdout and initial-seed replication; conditional rule-bootstrap intervals do not settle those questions.

2026-10-15 23:38 (+13:00) · 3d00123 · intent

The user requested a commit. The shared manuscript and its README/PDF contain many unrelated uncommitted edits. Stage only this comparison against HEAD, retaining the existing working versions. The committed source uses a provenance footnote in place of Audit 42 because the expanded audit appendix is unrelated uncommitted work. The staged manuscript compiles independently to 25 pages; its source/PDF hashes and reviewed pages are in results/staged-paper.json. The earlier 38-page PDF verification remains the record of the shared working manuscript at analysis time, rather than the isolated commit's PDF.