Skip to content

Iterate: versions and branches

A study that has already run is a result. Iterating on it must never silently destroy that result, and when the change is a counterfactual it must not re-pay for steps that cannot differ. SocioVerse gets both from one mechanism: every edit to a study becomes a version, and the subset of edits that touch only the environment after a fork step becomes a branch that inherits its parent's history by replay.

The edit you make is the state transition

A study's state is the tuple the pipeline writes to disk, and an edit is a pair a_k = (c, u) — the components c ⊆ {P, E, f, Θ} it touches, and the update u applied to them. Applying an edit produces the next version of the study:

σ     = (P, E_{0:T}, f, Θ)                        the study state

σ_{k+1} = h(σ_k, (c, u)) =
    σ_k                       c = {}         unchanged
    σ_k[ E_{t*:T} ← u ]       c = {E}        branch at fork step t*
    σ_k[ c ← u ]              otherwise      version
study state where it lives in the workspace
P population/population.json + population/roster.jsonl
E_0:T environment/environment.json (layers, scheduled_events, information_program.broadcasts)
f the study's own model.py / adapter/*.py, bound through simulation/simulation.json's decision_ref
Θ simulation/simulation.json (n_steps, seed, interaction_rounds, engine/store args)

h is implemented in skills/sv_workspace.py: create_version() for the third case, create_branch() for the second. Both append an entry to versions.json and freeze the previous state under versions/.

One word per thing

Branch is an environment-only, replay-inheriting version. Version is any other change, which runs cold from step 0. Fork means exactly one thing in SocioVerse — deriving a new study from a reference study, as below. A version is never called a fork, and a branch is never called a fork.

The rule: all changes go through /sv-iterate

Any change to an existing study — a new policy broadcast, more steps, a different population size, a tweaked threshold — goes through /sv-iterate.

Don't hand-edit and re-run

Editing artifacts in place and re-running overwrites the previous trajectory with no record. "add one more policy and run again" is an iteration, not an edit.

Before the gate: the reference guard

A study whose study.yaml carries reference: true — the bundled demo templates and adapted baselines — is fork-only. This guard runs before the version gate and bypasses it entirely: new version, branch and in-place all mutate the same study_id, which is exactly what a shared baseline must never suffer.

So a "modify this" request against a reference study forks it into a new study:

from skills.sv_workspace import fork_study, init_manifest, study_paths

fork = fork_study("studies", "<reference_id>", "<new_id>")   # rewrites study_id in every
                                                             # copied artifact, reference=false
init_manifest(fork)                                          # the fork starts its own v1 line
print("iterate on:", study_paths(fork)["study"])

fork_study() copies study.yaml, the resources/environment/population/simulation JSON and grounding/grounding.json, drops the source's trajectory/ and reports/ outputs, and refuses to overwrite an existing study_id. No code is copied — the fork keeps pointing at the source's registered provider_refs.

  • A fresh fork has no history to version, so the version gate is skipped. It starts its own version line at v1 and goes straight into the pipeline.
  • Editing the baseline template in place is a rare, deliberate choice that requires an explicit, spelled-out warning first: every future study built from that template inherits the change.

A non-reference working study skips this guard and goes straight to the gate.

The version gate: three exits

/sv-iterate always asks (never assumes) how to version the change. There are exactly three exits:

exit when it applies what the run costs
(a) new version (recommended default) any change to P, f or Θ, or to an environment entry that bites before the fork step cold run from step 0 — the full horizon is paid again
(b) branch (counterfactual) the edit only adds interventions at at_step >= t* to the environment bundle, or only stretches n_steps steps 0..t*-1 are replayed from the parent with no decision calls, only t*..n_steps spend budget
(c) modify in place throwaway fixes the user explicitly accepts losing the current outputs are overwritten on re-run and unrecoverable

Exit (a) has a sub-question: where to start from. It is asked when the request refers to an earlier version (or the user brings one up), not on every new version. Current version keeps the live artifacts as the starting point. Historical version v<X> restores that archived version's artifacts into the live directory first, and parent records where the new version came from. That is a starting point, not a branch — the run is still cold, and nothing is inherited from the parent's trajectory.

Mode and provider switches are new-version-only

A switch of decision/run mode or provider — scripted ↔ LLM, local ↔ real-LLM, deterministic ↔ stochastic — is not offered exit (c) at all. The earlier run is the scientific comparison baseline: the whole point of a scripted dry-run or a local-model pass is to sit next to the real one. Such a switch changes f, so exit (b) is out too. Only exit (a) is offered, with its starting-point sub-question. See LLM clients and dry runs.

Creating it: create_version and create_branch

from skills.sv_workspace import create_version, create_branch, current_version

current_version("studies/<id>")          # the parent version id

# (b) branch — only interventions at at_step >= J were added; 0..J-1 come from the parent
new_id = create_branch("studies/<id>", note="<one line>", fork_step=J)

# (a) new version from the current one — re-enter at the earliest affected stage
new_id = create_version("studies/<id>", note="<...>",
                        resume_from_stage="sv-build-environment")

# (a) new version starting from an older version's artifacts
new_id = create_version("studies/<id>", note="<...>", base="v1",
                        resume_from_stage="sv-init")

Signatures, as implemented:

create_version(study_dir, note, base=None, warm_start=None, resume_from_stage=None) -> str
create_branch(study_dir, note, *, fork_step, source=None,
              resume_from_stage="sv-build-environment") -> str
  • fork_step is keyword-only and must be >= 1 — step 0 is the materialized initial state, so there is nothing to inherit. create_branch raises ValueError otherwise.
  • create_branch records warm_start = {"source": src, "resume_from": fork_step - 1} and delegates to create_version, which sets kind = "branch" whenever a warm_start is present. (warm_start is the on-disk name of the replay spec and predates this vocabulary.)
  • source (branch) and base (version) default to the current version. Naming an older one restores that version's snapshot into the live directory first.
  • resume_from_stage defaults to "sv-build-environment" for a branch, because the environment bundle is the only artifact a branch re-authors. create_version defaults it to "sv-init" when base is not the current version, so an old design is re-reviewed stage by stage.
  • Both calls are crash-safe in this order: freeze the live directory into versions/v<current>/, restore the base if one was named, clear the live run outputs, then append the entry and move current (atomic, last).

Do not hand-edit simulation.json to set up a branch. /sv-run reads the manifest through resolve_warm_start(), which keeps versions.json the single source of truth.

Branch replay

A branch does not re-simulate its inherited prefix. LongitudinalSimulator.run() in socioverse/engine/loop.py sees a WarmStartSpec and calls _replay_inherited() before entering the real loop:

  1. P and E_0 are materialized from the same bundles and the same seed.
  2. For each step 1..t*-1 it activates the parent's schedule (env.advance_to(t)), reads the parent's stored rows for that step from its panel table, rebuilds them as Action objects with source="replay", applies them with env.apply(actions), pushes them into each agent's memory window, and records the panel, metrics and events rows into the new store.
  3. From t* on, decide_batch() runs as usual and the branch pays for its own steps.

Because a from-scratch study's env.apply(actions) is a pure function of the actions it is given, the replayed rows reproduce the parent's rows exactly, the environment reaches the same E_(t*-1), and every agent resumes with the memory window it had in the parent. Two branches off the same parent therefore share an identical realized history, which is what makes Δ_t = Y(E′) − Y(E) attributable to the declared intervention alone.

Eligibility

A branch needs a from-scratch (Path B) study with interaction_rounds == 1. Multi-round studies re-run from step 0: _replay_inherited() raises ValueError rather than replaying them incorrectly. That exclusion is not a limit of the replay code but of what was stored — only the final round's actions are persisted in the parent panel, so the intermediate rounds that produced them cannot be reconstructed. See Multi-round interaction.

Wrapped-legacy studies (the Chicago adapter) are excluded for a different reason: their env.apply() is not a pure function of the actions it is handed — the legacy move intent is set on hidden legacy agents during decide_batch() — so replaying stored actions would silently diverge. Nothing in the engine catches that, which is why /sv-iterate does not offer exit (b) for them at all.

_load_parent_panel() normally reads versions/<source>/trajectory/study.duckdb, and it also accepts a panel_live.jsonl path — the per-step stream an interrupted run leaves behind. Pointing a replay at that file resumes the same run from its last completed step instead of re-paying for it.

A worked example. The campus dining study ran 4 steps as v1, then branched at fork_step=3 into v2, which broadcasts a ¥100/month meal subsidy from month 3 and changes nothing else. v2 reproduced v1's rows for steps 0–2 exactly (mean satisfaction 0.6786 / 0.6581 / 0.6459 in both) and called the decision model only for steps 3–4, where satisfaction held at 0.6320 and 0.6174 against the parent's 0.6218 and 0.5970 — Δ_4 = +0.0204 against an identical history, with canteen share ending at 0.4867 rather than 0.4267. A third version, v3, is the degenerate case: it extends the horizon and adds nothing, so it branches off v2 at fork_step=5 and replays everything v2 ran.

The E-only invariant and automatic downgrade

A branch claims that everything differing from its parent after the fork step is caused by the declared intervention. check_branch_invariant() verifies that claim against the parent's frozen snapshot before any run, because a later hand edit can quietly break it.

what is compared rule
population/population.json identical, ignoring the volatile keys created_at / generated_at / updated_at / materialized_at / written_at
population/roster.jsonl byte-identical (SHA-256)
the study's own *.py and adapter/*.py byte-identical — the behaviour function f is code, and a branch that edits it is not a branch
simulation.json: seed, decision_ref, decision_args, collector_ref, collector_args, interaction_rounds, store_ref, store_args, engine_args identical. n_steps is deliberately exempt — a branch may run a longer horizon than its parent
the environment before the fork: provider_ref, provider_args, and every scheduled_events / broadcasts entry with at_step < fork_step identical — interventions may only be added at at_step >= fork_step
the parent's environment layers a branch may add a layer (for instance an information layer to carry its broadcast) but may never rewrite or delete one the parent already had

A violation downgrades the version, it does not raise. kind becomes "version", the replay spec is dropped, fork_step is cleared, the run goes cold from step 0 — the results stay correct, they just cost more — and the reason is written onto the manifest entry as downgraded_from_branch (source, fork_step, one violation line per offending artifact, at). resolve_warm_start() calls the check on every path to a run, so a downgraded branch simply has no replay left to resolve. The skill is required to tell you which artifact broke the branch and what recomputing the prefix adds to the bill.

Pre-check at any time without touching the manifest:

from skills.sv_workspace import check_branch_invariant

check_branch_invariant("studies/<id>", downgrade=False)
# {'kind': 'branch', 'ok': False, 'violations': [...], 'downgraded': False,
#  'source': 'v1', 'fork_step': 5, 'error': None}

A downgrade is not undone in place — the entry is a version now. To keep the branch, restore the offending artifact from the parent's snapshot and open a fresh one with create_branch.

The review gate (待复核)

Creating a version or a branch requires saying where the pipeline resumes. resume_from_stage is what the dashboard parks its pointer on and reviews forward from:

change resume_from_stage
branch sv-build-environment — create_branch sets it, do not pass it again
new version from the current one the earliest affected stage — a new broadcast → sv-build-environment, a population change → sv-build-population, a decision-logic change → sv-build-model
new version from a historical version sv-init — starting from an old design is a large context shift, so everything is re-reviewed

From that stage forward, every carried stage shows as 待复核 / pending review in the dashboard until you handle it. There are exactly two ways to handle one:

  • Affected stage → re-author it through its sv-* skill, as usual.
  • Unaffected stage → validate the carried artifact with validate_handoff(path, Model), confirm it still fits the new question, and post its narrative. That narrative is what clears the 待复核 flag.

Verify and narrate — do not rewrite an unaffected artifact

Rewriting a carried artifact "just to be safe" churns its mtime and corrupts the dashboard's carried-vs-stale detection. Validate it, say why it still holds, and leave the bytes alone.

A carried stage that is neither re-authored nor verified stays 待复核 and holds the pointer — so the pipeline can never look finished while an unreviewed stage rides along from the parent version. /sv-run still confirms before spending.

What lands on disk

versions.json at the study root is the manifest — {"current": "<id>", "versions": [...]} — written atomically (tmp + rename) so the dashboard poller never reads a torn file. One entry per version:

{
  "id": "v2",
  "parent": "v1",
  "note": "media campaign at step 5",   // ≤ ~50 chars, shown in the dashboard version box
  "kind": "branch",                      // "branch" | "version"
  "fork_step": 5,                        // the branch's first self-computed step; null for a version
  "warm_start": {"source": "v1", "resume_from": 4},   // the replay spec; null for a version
  "resume_from_stage": "sv-build-environment",
  "created_at": 1758000000.0,
  "snapshot": null,                      // null while this version is live; becomes
                                         // "versions/v2" when the NEXT bump freezes it
  "downgraded_from_branch": {            // only present after an invariant violation
    "source": "v1", "fork_step": 5, "violations": ["..."], "at": 1758000000.0
  }
}

v1 is a plain kind: "version" with parent: null — there is no separate root kind. Manifests written before kind and fork_step existed still classify correctly: version_kind() and version_fork_step() fall back to the warm_start provenance, so no migration is needed.

versions/<id>/ is the frozen workspace of that version: a full copy of the live directory including code, bundles, trajectory/study.duckdb, reports/ and narrative/. Excluded from the copy are versions/ itself, versions.json, runs/, __pycache__, .DS_Store, and the two transient run files progress.jsonl and panel_live.jsonl — an archive must never look mid-run to the dashboard. The archived study.duckdb is exactly what a child branch replays. versions/ is gitignored, like run outputs: old versions never auto-delete.

The live directory loses trajectory/ and reports/ on every version bump (they survive in the snapshot), so the new version starts honestly at "built, not yet run".

The dashboard reads all of this through versions_view() in dashboard/server/app.py, which returns per version: kind, fork_step, branch_of (the warm_start.source) and downgraded. See The run dashboard.

Where to go next