It gets better every night
Evolution round 20261012-1734 decided ($0.37): policy tripped integrity flags; playbook -0.91 (not adopted); playbook tripped integrity flags · 42m ago
Evolution round 20261012-1734: strategy s-20261012-1734 (new, written by the meta level); 3 candidate(s) passed the static gates (playbook, policy, playbook); 6 benchmark run(s) queued, budget $1.50 · 2.3h ago
Evolution round 20261012-1638 decided ($0.32): playbook -2.95 (not adopted); policy -3.68 (not adopted); playbook -3.98 (not adopted) · 2.3h ago
Evolution round 20261012-1638: strategy s-20261012-1638 (new, written by the meta level); 3 candidate(s) passed the static gates (playbook, policy, playbook); 6 benchmark run(s) queued, budget $1.50 · 3.2h ago
| playbook | attempts | built | build rate |
|---|
| gen 1 | 2 | 0 |
0% |
| gen 2 | 60 | 31 |
52% |
| gen 5 | 2 | 1 |
50% |
Current playbook generation 5: 16 learned rules. A rule is adopted only if
its regex matches its own failure class and almost nothing else, and its advice passes an integrity filter (no hardcoding,
no skipping verification, allowlisted hosts only) and the new version beats the old one on a real benchmark run.
Held-out papers never feed the learner.
- missing_module_autofix [missing_module] Install the missing module in one turn: `pip install -q <mod>`; add fallbacks `xlrd openpyxl` for Excel, `opencv-python-headless` plus libxcb for cv2. Re-run th
- heavy_pip_install_timeout [timeout] Heavy wheels (torch/pyvista/CUDA/h5py) can exceed the cell timeout. Install one package per cell with `--no-cache-dir`; check `python3 -c "import torch"` first
- raw_data_absent_placeholder [data_missing] Missing raw data must be fetched during setup while network is open. Run the repo's download/prepare script early. If the source host is allowlisted (UCI, Zenod
- relative_import_run_as_module [python_error] Run package files as modules: `cd /work/repo && python -m pkg.module`, or call the repo's CLI/entry script. Executing a file that sits inside a package breaks r
- prefer_repo_entrypoint_end_to_end [late_give_up] As soon as deps import, run the repository's own end-to-end entry point (train/eval script or paperci_run.sh) at reduced scale instead of re-creating notebook c
- lb_egress_proxy_403_prefetch_fallback [egress_denied] Fetch data/weights during setup, never at run time. If the proxy still returns 403 Filtered (even for an allowlisted host), try the mirror or the repo's cached
- lb_missing_system_shared_lib [data_missing] Missing OS C libraries block cv2/PyQt imports. Run `apt-get update && apt-get install -y libxcb1 libgl1 libglib2.0-0 libsm6 libxext6 libxrender1`, or install `o
- lb_reduced_cpu_verifier_defer_heavy_claims [late_give_up] When key metrics need GPU, API keys, or gated assets, don't quit empty-handed. Write a small standalone numpy script reproducing one core CPU-checkable claim at
Self-improvement · tested before adopted
20261012-1734 via s-20261012-1734: policy tripped integrity flags; playbook -0.91 (not adopted); playbook tripped integrity flags · $0.37
20261012-1638 via s-20261012-1638: playbook -2.95 (not adopted); policy -3.68 (not adopted); playbook -3.98 (not adopted) · $0.32
20261012-1524 via s-20261012-1524: playbook tripped integrity flags; policy rejected by the static gate; playbook -3.42 (not adopted) · $0.47
20261012-1426 via s-20261012-1426: policy tripped integrity flags; logbook playbook +2.04 adopted as generation 5; playbook -0.07 (not adopted) · $0.39
20261012-1309 via s-20261012-1309: playbook -0.00 (not adopted); policy -3.36 (not adopted); playbook -4.18 (not adopted) · $0.44
| improver strategy | used | mean gain | |
|---|
| s-turns | 2 | +4.74 | Optimise for reaching a working /work/paperci_run.sh in fewer turns: prefer rules and cont |
| s-20261012-0350 | 2 | +4.58 | Optimise for the agent reaching a real /work/paperci_run.sh run on true repo code in the f |
| s-20261012-1426 | 2 | +2.46 | Prefer one short rule that cuts dead discovery and setup turns so the agent runs the repo' |
| s-20261012-1734 | 2 | +0.71 | Prefer controller settings and one-sentence rules that cut exploration before the run: pic |
| s-env | 0 | +0.00 | Optimise environment setup: dependency resolution, CPU-only fallbacks and downloading data |
| s-20261012-0804 | 0 | +0.00 | Optimise for the fewest turns to a real /work/paperci_run.sh run on true repo code: cut di |
| s-base | 1 | -1.02 | (base improver) |
| s-20261012-1238 | 1 | -5.00 | Prefer one focused rule that shortens the path from failure cluster to a real run of the r |
Every night the improver proposes a new playbook and a rewritten control policy. Each is run for real on a fixed dev benchmark against the current version and adopted only if it scores higher with no regression and no integrity flag. Code changes also need a human approval, and adopted changes are re-checked on live papers and rolled back if worse. The improver's own strategy is scored the same way (the recursive part). The verifier, sealed claims and cost limits are fingerprinted; any change voids the round.