One step, five phases¶

The longitudinal simulation loop is one class: LongitudinalSimulator in socioverse/engine/loop.py. Its run() method is the whole runtime — everything else in socioverse/ exists to be called from it. This page walks one step of that loop at the level of function calls, typed objects and the rows they leave in the store.

What a step computes¶

A step advances the environment by one unit of the study's time_unit and produces exactly one behaviour per agent. Written out:

e_t^i   = ω(E_t, p_i)                 # observe: project the world onto agent i
b_{i,t} = f(p_i, e_t^i, m_{i,t})      # decide:  one typed action per agent
B_t     = { b_{i,t} }_{i=1..N}        # the step's collective behaviour
E_{t+1} = g(E_t, B_t)                 # apply:   fold the behaviours back in

ω is the observation operator, f the behaviour function, g the environment transition, and m_{i,t} = μ(b_{i,t−W:t−1}) the agent's bounded memory of its own last W actions. Each symbol is a method on one of the four interfaces in socioverse/abc/:

symbol	interface method	implemented by
`P`	`PopulationProvider.build(seed) -> list[Persona]`	the study's population provider
`E_0`	`EnvironmentProvider.reset(seed)`	the study's environment provider
exogenous change	`EnvironmentProvider.advance_to(t) -> list[ScheduledEvent]`	same
`ω`	`EnvironmentProvider.observe_batch(agent_ids, t, round_idx) -> list[Observation]`	same
`f`	`DecisionModel.decide_batch(obs, memories) -> list[Action]`	the study's decision model
`g`	`EnvironmentProvider.apply(actions) -> None`	the study's environment provider
the panel row's state	`EnvironmentProvider.agent_state(agent_id) -> dict`	same
the metrics row	`MetricCollector.collect(env, actions, t) -> dict`	the study's collector
persistence	`TrajectoryStore.record / record_metrics / record_events`	`DuckDbTrajectoryStore`

The engine never reaches inside a provider. It only calls these methods, in a fixed order, and writes whatever they return.

Step 0 is not a step¶

def materialize_initial(self) -> tuple[list[Persona], list[TrajectoryRecord]]:
    personas = self.population.build(self.seed)
    self.env.reset(self.seed)
    panel0 = self._panel_rows(personas, actions=None, t=0)
    return personas, panel0

run() opens with materialize_initial(). It materializes the fixed population P — persistent, deterministic agent_ids that are never reused — builds E_0, and writes one panel row per agent at t = 0 with action_kind = NULL. No decision model is called at step 0 and no LLM token is spent. A step-0 metrics row is collected too, so every metric series starts at its pre-run baseline.

Two consequences worth knowing:

The method is a pure function of seed for deterministic providers, which is why sv-build-population can call the module-level twin socioverse.engine.builder.materialize_initial(env_bundle, pop_bundle, seed) to write population/roster.jsonl before any run exists. The roster you inspect at build time is byte-for-byte the t = 0 the run will start from.
A trajectory of n_steps: 12 holds 13 panel rows per agent (steps 0…12).

run() allocates one AgentMemory per agent as soon as materialize_initial() returns — before the store is even opened — and then enters the loop. A branch enters it later: _replay_inherited first re-derives steps 1 … t*−1 from the parent version's stored actions, and the first step decided for real is the fork step t*. Every computed step runs these five phases, in this order:

flowchart LR
    A["① exogenous update<br/>env.advance_to(t)"] --> B["② observe<br/>env.observe_batch(ids, t, r)"]
    B --> C["③ decide<br/>decision.decide_batch(obs, memories)"]
    C --> D["④ apply<br/>env.apply(actions)"]
    D -->|"r &lt; interaction_rounds-1"| B
    D --> E["⑤ record<br/>memory · panel · metrics"]

① Exogenous update¶

fired = self.env.advance_to(t)                     # ① exogenous update
if fired:
    self.store.record_events(t, [self._event_note(e) for e in fired])

Reads the environment bundle's scheduled_events and information_program.broadcasts (environment/environment.json). Writes the provider's own layer state, and one row per fired item into the events table.

This is the only phase that changes E without an agent doing anything, and it runs before anyone observes — so every agent in step t decides against the same world state. An intervention is nothing more than an entry that fires here: a ScheduledEvent (a typed numeric mutation — set, add, multiply, add_pct, replace_source — on a selected part of a declared physical layer) or a Broadcast (information content with a channel, a ttl and an audience that is either "all" or a selector dict). See Intervention and counterfactual branching.

The provider decides what the returned list contains. The engine logs whatever comes back, turning each item into a note via ev.note or f"{ev.op} {ev.property_name or ''}".strip(). Providers that want broadcasts in the events table wrap them in a ScheduledEvent carrying the note, as the Chicago adapter does:

newly = self.info_env.advance_to(t)                # InformationEnvironment
for b in newly:
    fired.append(ScheduledEvent(at_step=t, target_layer="information",
                                op="set", note=f"broadcast: {b.content}"))

The same notes are joined into the step's metrics row under the events column (_collect), which is what puts the vertical markers on a metric timeline.

② Observe¶

obs = self.env.observe_batch(agent_ids, t, r)      # ② observe

Reads the provider's layer state plus each agent's position, network and attributes. Writes nothing — it returns list[Observation], one per agent, and the engine passes it straight to the decision model.

An Observation (socioverse/schemas/runtime.py) is the four-quadrant view, modality × scope:

field	modality × scope	typical content
`macro_physical`	physical × macro	citywide vacancy rate, index level, weather
`local_physical`	physical × local	own tract's rent and composition, neighbours' opinions
`macro_information`	information × macro	a broadcast with `audience="all"`
`local_information`	information × local	an audience-scoped notice, neighbours' posts

Plus agent_id, step, and an optional rendered string — a pre-rendered prompt fragment, so an LLM decision model does not have to re-serialize the same dicts for every agent. Observation.quadrants_nonempty() returns which quadrants actually carry content, which is the quickest audit of whether a study's information layers are reaching anyone.

Three reusable layer helpers live in socioverse/env_layers/ and are composed by the provider — the engine never calls them:

InformationEnvironment (information.py) — activates and expires broadcasts by at_step/ttl, and renders them into macro() / local_for(agent_ctx) / rendered_lines(agent_ctx). Local audiences are resolved by a matcher, default match_selector, which understands selector keys of the form <attr>, <attr>_in, <attr>_below, <attr>_above, geoid_list and index_below. An empty selector matches nobody.
NeighborFeed (neighbor_feed.py) — turns post / reply actions into other agents' local_information, mediated by a MessageBus and a neighbors_fn, never O(N²) direct calls.
news_broadcast.py — default_information_layers() plus the macro_news(...) and local_notice(...) constructors. A citywide announcement and a ward notice are the same mechanism differing only in audience.

③ Decide¶

actions = self.decision.decide_batch(obs, self.memories)   # ③ decide: B_t = f(P, E_t)

Reads the observations and memories (dict[agent_id, AgentMemory]). Writes nothing — it returns list[Action], the step's B_t.

The interface is deliberately batch-first: decide_batch sees every agent at once, so an implementation is free to group agents by cohort, issue one LLM call per archetype, or run a thread pool over per-agent prompts. The engine never calls the model agent by agent. The shipped from-scratch templates use the thread pool — one LLM call per agent, sent concurrently — and never ask for one JSON reply covering the whole population; a legacy adapter keeps its engine's own pattern. (DecisionModel.decide() exists as a single-agent convenience and just wraps decide_batch.)

Nothing forces an implementation to return an action for every agent. Agents with no action that step simply get a panel row whose action_kind is NULL — which is also how replay reconstructs an idle agent.

④ Apply¶

self.env.apply(actions)                            # ④ apply: E_{t+1} = g(E_t, B_t)

Reads the whole batch of actions. Writes the provider's layer state — this call is g.

The engine hands apply the complete B_t and calls it once per round, so the round settles synchronously: no agent sees another agent's action inside its own decision, and there is no ordering advantage from being early in the list. A provider is free to resolve contention here — the Chicago adapter applies a per-race move budget and then annotates each action's payload with whether the move actually executed:

a.payload["moved"] = bool(agent.moved_this_step)
if agent.moved_this_step:
    a.payload["new_tract"] = agent.tract_id

That mutation happens before the panel row is built, so the panel records the realized outcome, not just the intent.

For from-scratch (Path B) studies apply is expected to be a pure function of the actions passed to it. That single property is what makes replay — and therefore branching — possible.

⑤ Record¶

for a in last_actions:                             # ⑤ record — memory window first
    self.memories[a.agent_id].push(t, a)

panel_t = self._panel_rows(personas, last_actions, t)
self.store.record(panel_t)
self._emit_panel(panel_t, t)
m = self._collect(personas, last_actions, t, fired_notes=[...])

Reads env.agent_state(agent_id) for every persona plus the step's actions. Writes, in this order: each agent's memory window, one panel row per agent, and one metrics row.

build_panel_rows (a free function, so the pre-run materializer and the loop produce identically shaped rows) builds each TrajectoryRecord as:

TrajectoryRecord(agent_id=p.agent_id, step=t,
                 state=env.agent_state(p.agent_id),
                 action_kind=a.kind if a else None,
                 action_payload=a.payload if a else {})

Note what is not in the panel: rationale and source stay on the in-memory Action and are not persisted by the store. The panel keeps exactly kind + payload — the minimum a replay needs to rebuild the action.

_collect calls collector.collect(env, actions, t), forces m["step"] = t, appends the fired-event notes as m["events"], hands the row to store.record_metrics, and finally fires the per-step heartbeat and the optional on_step(t, metrics) callback.

Two plain files are written next to the DuckDB store as a side effect, both best-effort and never able to break a run: progress.jsonl (one line per step, truncated at t = 0) and panel_live.jsonl (the same rows as the panel table). They exist because DuckDB is single-writer-locked while a run is open, so the dashboard reads these instead of the database mid-run.

The Action contract¶

class Action(BaseModel):
    agent_id: str
    step: int
    kind: str                                       # move | stay | post | reply | adopt_opinion | ...
    payload: dict[str, Any] = {}
    rationale: list[str] = []
    source: Literal["llm", "rule", "fallback", "replay"] = "llm"

kind — the action type, a study-defined string. The environment branches on it (if a.kind == "update_opinion": ... in studies/opinion_diffusion/model.py); NeighborFeed.ingest routes only "post" and "reply" onto the bus.
payload — the type-specific content: a destination tract, a number, a text body. Must be JSON-serializable; it is stored as a JSON column.
rationale — a list of first-person reason strings. Useful for auditing an LLM run; not persisted to the panel.
source — exactly four values:

`source`	set when
`llm`	the model replied and the reply parsed
`rule`	a deterministic update rule produced the action
`fallback`	the LLM was called but its reply failed to parse, so the rule filled in
`replay`	the engine rebuilt the action from a parent version's panel rows

Keeping fallback distinct from rule is what lets you measure how much of an "LLM run" was actually the LLM.

Because all four behaviour functions return this same typed object, the environment and the store are decoupled from the mechanism that produced the decision. apply, agent_state, the panel schema and every metric work unchanged whether f was a rule, a prompt, a trained policy, or a replay — which is also what makes a rule-versus-LLM comparison a controlled one: both act in the same discrete action space and are recorded in the same table.

The four behaviour functions f¶

f is the only place a study's mechanism lives, and inside the loop it is always a DecisionModel. A from-scratch (Path B) study registers it under a string key:

from socioverse.engine.registry import register

@register("decision", "opinion.decision")
class BoundedConfidenceDecision(DecisionModel):
    def __init__(self, confidence: float = 0.3, llm_kind: str = "scripted", ...):
        ...

simulation/simulation.json names that key and its constructor arguments:

{
  "decision_ref": "opinion.decision",
  "decision_args": {"confidence": 0.3, "llm_kind": "openai", "model": "gpt-4o-mini", "max_workers": 8}
}

build_simulator (socioverse/engine/builder.py) resolves it with resolve("decision", sim_config.decision_ref)(**sim_config.decision_args). The registry holds six kinds — environment, population, decision, collector, store, reporter — and resolution fails loudly if the study module was never imported, because @register runs at import time.

A study that wraps a legacy simulator skips the registry for this one component: its own build_*_simulator factory constructs the decision model directly (decision = SchellingDecisionModel(engine) in studies/chicago_schelling/adapter/build.py) so several components can share one mutable legacy engine. The loop cannot tell the difference — it receives a DecisionModel either way.

`f`	what it does	how it is wired
rule-f	evaluates a classical update rule; the parity baseline	one registered `DecisionModel` whose `decide_batch` runs the arithmetic and tags `source="rule"`
LLM-f	composes persona + the four observation fields + the memory window into a prompt, parses the structured reply into an action	usually the same registered class, switched by a `decision_args` flag (`llm_kind: "scripted" \\| "openai"`), so the rule stays available as the fallback path
RL-f	maps observations to actions through a trained policy, one per class of strategic actor	enters through a study adapter (`studies/drug_procurement/adapter/`, a study not included in the open-source release) that runs the external trainer and exports the same `panel` / `metrics` tables
hybrid-f	different implementations for different agent groups in one run — typically LLM-f for a small core, rule-f for the remainder	one registered class that branches inside `decide_batch`, e.g. `studies/hisim_roe/model.py`'s `HybridTwitterDecision`: `self._decide_core(ob, ...) if is_core(ob.agent_id) else self._decide_ordinary(ob)`

Pairing rule-f and LLM-f behind one decision_ref is the pattern to copy: the LLM path falls back to the identical rule arithmetic whenever a reply fails to parse, so the metric columns are always filled and the two runs are comparable by construction.

The memory window¶

class AgentMemory:
    def __init__(self, agent_id: str, window: int = 8):
        self.history: deque[dict] = deque(maxlen=window)

    def push(self, t, action=None, observation_summary=None):
        self.history.append({"t": t, "action_kind": ..., "payload": ..., "summary": ...})

Each agent owns a bounded rolling window of its own actions — not other agents', not the environment's. The default is 8 steps — the W of m_{i,t} = μ(b_{i,t−W:t−1}) above — configurable per study via engine_args.memory_window, which build_simulator reads as sim_config.engine_args.get("memory_window", 8):

{"engine_args": {"memory_window": 12}}

run() creates the windows before any step is computed and pushes into them in phase ⑤. AgentMemory also offers recent(n), last_action_kind() and count_action(kind) so a decision model can condition on the agent's own past ("I already moved twice, I will stay") without re-reading the store.

The window is filled only from the actions the panel records, which makes it exactly reconstructible from the panel table. That is the precondition for a branch: _replay_inherited rebuilds the parent's actions from its panel rows, applies them, and pushes them into the windows, so at the fork step each agent resumes with precisely the memory it had in the parent run.

out.append(Action(agent_id=r["agent_id"], step=t, kind=kind,
                  payload=dict(r.get("action_payload") or {}), source="replay"))

`interaction_rounds`¶

for r in range(self.interaction_rounds):
    obs = self.env.observe_batch(agent_ids, t, r)
    actions = self.decision.decide_batch(obs, self.memories)
    self.env.apply(actions)
    last_actions = actions

interaction_rounds (default 1, clamped by max(1, …)) is the number of observe → decide → apply cycles inside one step. Raising it lets agents react to what others said within the step; advance_to still fires only once, before round 0, and rounds are not time.

Only the final round is recorded — so a multi-round study cannot be replayed

Memory receives last_actions, and the panel row carries the last round's kind + payload. Rounds 0 … K−2 leave no trace in the trajectory. That is why _replay_inherited refuses to run when interaction_rounds != 1:

ValueError: branch replay supports interaction_rounds == 1 only (study has 3);
the parent panel keeps only the final round's actions, so multi-round steps
can't be replayed exactly — re-run from step 0.

(warm_start is the engine's field name for a branch's replay spec — SimulationConfig.warm_start, a WarmStartSpec whose fork_step is resume_from + 1.) A multi-round study therefore iterates as a new version and runs cold from step 0, never as a branch. See Multi-Round Interaction and Intervention and counterfactual branching.

What lands on disk¶

DuckDbTrajectoryStore (socioverse/io/duckdb_store.py) writes one file per run, studies/<id>/trajectory/study.duckdb. open() deletes an existing file first — a run is always fresh; older results live in version snapshots.

table	schema	written by	cardinality
`panel`	`agent_id VARCHAR, step INTEGER, state JSON, action_kind VARCHAR, action_payload JSON`	`record()`, streaming per step	`N × (T + 1)` rows
`metrics`	`step` first, then one column per metric key, plus an `events` column if any step fired something (NULL on the rest)	buffered by `record_metrics()`, materialized as a wide table at `finalize()`	one row per step
`events`	`step INTEGER, note VARCHAR`	`record_events()`, streaming	one row per fired event or broadcast
`messages`	`author_id, step, round, channel, content, audience`	`record_messages()`, table created lazily on first call	only for studies that persist a bus

(agent_id, step) identifies a panel row — the longitudinal key, held by construction rather than by a declared constraint. finalize() also exports panel.parquet and metrics.parquet beside the database.

Query it with any DuckDB client, or from Python:

from socioverse.io import open_readonly

con = open_readonly("studies/opinion_diffusion/trajectory/study.duckdb")

One agent's complete trajectory — state, action and payload at every step:

SELECT step,
       state->>'opinion'           AS state_opinion,
       action_kind,
       action_payload->>'opinion'  AS decided_opinion
FROM panel
WHERE agent_id = 'od-007'
ORDER BY step;

The metric series with the interventions that fired against it:

SELECT m.*, e.note
FROM metrics m
LEFT JOIN events e USING (step)
ORDER BY m.step;

How many agents acted, by action kind, per step:

SELECT step, action_kind, count(*) AS n
FROM panel
WHERE action_kind IS NOT NULL
GROUP BY step, action_kind
ORDER BY step, n DESC;

The metrics columns are whatever the study's MetricCollector.collect() returned — step, mean_opinion, opinion_std, frac_above_0_5, events for the opinion-diffusion template — so SELECT * first. collector.columns() is the declared contract when a study bothers to declare one.