socioverse.io — the trajectory store

The durable, queryable panel store: tables panel, metrics, events (and messages when the bus is used), with parquet exports at finalize.

socioverse.io.duckdb_store

DuckDbTrajectoryStore — database-like panel/metrics/events/messages store.

One DuckDB file per study (trajectory/study.duckdb) with tables: - panel(agent_id, step, state JSON, action_kind, action_payload JSON) -- longitudinal panel - metrics(step, ) -- per-step aggregates - events(step, note) -- fired interventions - messages(author_id, step, round, channel, content, audience) -- inter-agent (scenario 2)

Panel rows + events + messages stream in per step (so a reader can attach mid-run); each batch is written in ONE transaction (DuckDB commits every statement separately in autocommit mode, which costs one fsync per agent row). The wide metrics table is materialized at finalize(). Parquet copies are exported for portability. The reporter (and any later real-time viewer) just opens the same file and runs SQL — e.g. SELECT step, state->>'tract_id' FROM panel WHERE agent_id = ?.

DuckDbTrajectoryStore

DuckDbTrajectoryStore(
    db_path: str | Path,
    study_id: str = "study",
    export_parquet: bool = True,
)

Bases: TrajectoryStore

Source code in socioverse/io/duckdb_store.py

def __init__(self, db_path: str | Path, study_id: str = "study", export_parquet: bool = True):
    self.db_path = Path(db_path)
    self.study_id = study_id
    self.export_parquet = export_parquet
    self._con: duckdb.DuckDBPyConnection | None = None
    self._metrics_rows: list[dict[str, Any]] = []
    self._messages_created = False

open_readonly

open_readonly(
    db_path: str | Path,
) -> duckdb.DuckDBPyConnection

Open a finalized study store for querying (reporter / interactive viewer).

Source code in socioverse/io/duckdb_store.py

def open_readonly(db_path: str | Path) -> duckdb.DuckDBPyConnection:
    """Open a finalized study store for querying (reporter / interactive viewer)."""
    return duckdb.connect(str(db_path), read_only=True)