AwsIncidentsBench · T01 · recorded run replay

The Empty Ledger Endpoint

At 02:31 a race in endpointd deleted every DNS record for ledger.internal, and all 30 checkout hosts cached the miss for 15 minutes. Replay a recorded agent run, or the scripted reference, on the run's own exported world: each house is one core host and recovers when that run's export says it did.

ET02:44:00
checkout4.0%
webhook backlog0
writer load0
Loading
Agentcalls 0 of 45
not paged yet
The page fires at 02:40.
resolves ledger.internal NXDOMAIN cached reconnecting request to ledger-writer agent call
space play · ←/→ 1 min · shift 10 min · [ ] steps · drag to orbit

Replayed from the run's own exports, not a simulation: the agent's calls from the aws server's meter, each host's recovery from its DNS export, the world's series, the colleagues' lines and the grader's rows. Methods below says how.