Problem
Concordia currently has no standardized observability layer for simulation runs.
Researchers and engineers face three concrete problems:
1. No reproducible audit trail
When a simulation completes, there is no structured record of which component
produced which context, what the LLM was prompted with at each step, and what
decision was made. Debugging unexpected agent behavior requires manually
instrumenting every component with print statements.
2. No component-level telemetry
ProfiledLanguageModel exists but operates in isolation — there is no way to
attribute token costs or latency to a specific component (e.g., "AssociativeMemory
consumed 3x more tokens than expected on step 12"). For research and production
usage, this makes cost management and bottleneck identification impossible.
3. No simulation replay / partial recovery
If a long multi-agent simulation fails mid-run (LLM timeout, ThreadPoolExecutor
crash, OOM), all intermediate state is lost. There is no checkpoint/resume
mechanism, and no way to "replay" a simulation from a saved event log.
Impact
These gaps directly affect Concordia's stated use cases:
- AI safety research: Requires full auditability of agent reasoning chains
- Social science: Requires reproducible simulation records for publication
- Economic modeling: Requires per-step state snapshots for analysis
- Synthetic data generation: Requires structured logs to validate data quality
Proposed Solution
Add a lightweight SimulationLogger abstraction with the following interface:
class SimulationEvent:
step: int
agent_name: str
phase: str
component_name: str
prompt: str | None
output: str
token_usage: dict | None
timestamp: float
class SimulationLogger(abc.ABC):
def log_event(self, event: SimulationEvent) -> None: ...
def flush(self) -> None: ...
Built-in implementations:
JsonLinesLogger — writes newline-delimited JSON to a file (replay-friendly)
InMemoryLogger — for testing and notebooks
NullLogger — zero overhead default (backward compatible)
The engine's _parallel_call_ and EntityAgent lifecycle hooks would emit
events to an injected logger. This is purely additive — existing code needs
zero changes.
Why This Matters
This is a foundational observability primitive that every serious Concordia
user needs but currently has to build themselves. A standardized interface
means community tooling (dashboards, replay tools, cost analyzers) can be
built on a stable API.
Problem
Concordia currently has no standardized observability layer for simulation runs.
Researchers and engineers face three concrete problems:
1. No reproducible audit trail
When a simulation completes, there is no structured record of which component
produced which context, what the LLM was prompted with at each step, and what
decision was made. Debugging unexpected agent behavior requires manually
instrumenting every component with print statements.
2. No component-level telemetry
ProfiledLanguageModelexists but operates in isolation — there is no way toattribute token costs or latency to a specific component (e.g., "AssociativeMemory
consumed 3x more tokens than expected on step 12"). For research and production
usage, this makes cost management and bottleneck identification impossible.
3. No simulation replay / partial recovery
If a long multi-agent simulation fails mid-run (LLM timeout, ThreadPoolExecutor
crash, OOM), all intermediate state is lost. There is no checkpoint/resume
mechanism, and no way to "replay" a simulation from a saved event log.
Impact
These gaps directly affect Concordia's stated use cases:
Proposed Solution
Add a lightweight
SimulationLoggerabstraction with the following interface:Built-in implementations:
JsonLinesLogger— writes newline-delimited JSON to a file (replay-friendly)InMemoryLogger— for testing and notebooksNullLogger— zero overhead default (backward compatible)The engine's
_parallel_call_andEntityAgentlifecycle hooks would emitevents to an injected logger. This is purely additive — existing code needs
zero changes.
Why This Matters
This is a foundational observability primitive that every serious Concordia
user needs but currently has to build themselves. A standardized interface
means community tooling (dashboards, replay tools, cost analyzers) can be
built on a stable API.