AgentReplay: Deterministic Replay and Counterfactual Debugging for AI Agent Reliability

Agent reliability is the least mature research area in applied AI today — mature enough to matter commercially, immature enough that its foundational science is still being written in real time. Papers with titles like "Towards a Science of AI Agent Reliability" (Rabanser, Kapoor, Narayanan et al., ICML 2026) and ReliabilityBench exist precisely because the industry does not yet have good tools for measuring reliability, or even a settled definition of it.

This whitepaper introduces AgentReplay — an open-source deterministic replay and counterfactual-debugging layer that lets engineering teams reproduce, inspect, and patch a specific agent failure bit-for-bit, without spending another token on inference. The library is live at github.com/gadda00/agentreplay with 99 passing tests, 0.67% recording overhead, and 100% reproduction fidelity on the synthetic validation set.

1. Executive Summary

This work evaluates three scoped product opportunities in the agent-reliability space — a scaffolding-normalized benchmark harness, a deterministic replay/checkpoint layer for nondeterministic agent chains, and cost-bounded planning middleware — and recommends building the second: AgentReplay.

The case for this choice rests on three converging facts. First, agents fail unpredictably and often irreproducibly. Second, the tools teams already use (LangSmith, Langfuse, AgentOps, Laminar, LangGraph's own "Time Travel") give visibility into what happened but stop short of guaranteeing the same thing happens again. Third, inference cost is now cited as enterprises' single biggest blocker to scaling agents — which a genuinely deterministic replay layer directly relieves, because replaying a recorded run costs zero additional model calls.

2. Research Foundation: The State of Agent Reliability in 2026

2.1 Reliability does not yet have an accepted science

Through most of 2024–2025, agent progress was reported almost entirely through accuracy: the percentage of benchmark tasks completed. In February 2026, a multi-institution paper — co-authored by Sayash Kapoor and Arvind Narayanan of Princeton's "AI as Normal Technology" project alongside independent researcher Stephan Rabanser — argued explicitly that accuracy and reliability are different axes, and that the field had been conflating them. Presented as an ICML 2026 poster, the paper proposes decomposing reliability into four measurable dimensions grounded in safety-critical engineering practice — consistency, robustness, predictability, and safety — rather than a single pass rate.

Applying these metrics across twelve frontier models, the authors report a striking finding: reliability gains have been essentially flat over 24 months of model releases even as accuracy climbed, and the plateau is industry-wide rather than tied to any one vendor. Their motivation is not abstract — the paper opens by citing well-documented real incidents (a coding assistant that deleted a production database despite explicit instructions not to; an autonomous purchasing agent that bypassed a confirmation step) as evidence that benchmark-capable agents keep failing in ways that internal evaluations did not anticipate.

2.2 The capability curve is real, and it outruns the reliability curve

METR's time-horizon research is the most cited empirical anchor in this space. The headline metric is the length of task — measured in the time a skilled human would need — that a frontier agent can complete with 50% reliability. METR's original analysis found this horizon has been doubling roughly every seven months across six years of data; a January 2026 update refined this further, finding the doubling time has compressed to roughly 4.3 months for the 2023-onward window. As of the most recent published measurements, Claude Opus 4.6 reached a 50% time horizon near 12 hours.

The mechanism behind the curve matters more than the curve itself for a reliability product. Independent analyses show the doubling pattern is well explained by a simple constant hazard rate: the agent has some fixed per-minute probability of failing, and success probability decays exponentially with task length, exactly like component failure in reliability engineering. This has two direct implications for a replay/debugging product: first, failures are not rare edge cases to be triaged occasionally — they are a statistically predictable, load-bearing feature of long-horizon agent operation. Second, the hazard rate varies substantially by domain, so a reliability tool has to be domain-agnostic at the recording layer.

2.3 The measurement problem: benchmarks are not comparing like with like

GAIA is a fixed set of 466 real-world, multi-step assistant tasks. As of mid-2026, published scores on the same 466 tasks range from roughly 44.8% up to 92.36% depending entirely on what scaffolding is permitted. The spread between the bare-model and system-level numbers on identical tasks is larger than the gap between most successive frontier model releases — meaning that, without knowing which scaffold produced a number, the number is close to meaningless for deciding whether a new technique actually helped.

This measurement instability compounds the debugging problem this work targets: if a team cannot even trust that a benchmark score reflects a fair comparison, it certainly cannot trust that a single production failure, observed once, generalizes to anything — unless that specific failure can be reproduced on demand and studied directly.

2.4 The business cost: cancellations, and inference as the dominant budget line

Gartner's June 2025 forecast predicts that over 40% of agentic AI projects will be canceled by the end of 2027, attributing this explicitly to escalating costs, unclear business value, and inadequate risk controls. A DigitalOcean survey of over 1,100 developers found that 44% of respondents report spending 76–100% of their entire AI budget on inference alone, and that 49% named inference cost at scale as their number-one blocker to scaling AI.

This is the direct commercial link between "agents are unreliable" and "agents are expensive": every time a team cannot understand why an agent failed without re-running it live, they are paying inference cost to investigate a problem that inference cost helped cause.

2.5 The observability gap: visibility without reproducibility

A mature tooling ecosystem already exists for watching agents run — LangSmith, Langfuse, AgentOps, Laminar, Arize Phoenix, Braintrust, and others all capture traces of LLM calls, tool invocations, and multi-step state. Independent 2026 benchmarking of four of these platforms on an identical multi-agent workload found real, measurable instrumentation overhead: LangSmith close to 0%, Laminar around 5%, AgentOps around 12%, and Langfuse around 15%.

Critically, none of these tools solve reproducibility in the strict sense a debugger needs. LangGraph's "Time Travel" feature snapshots graph state after every node and lets a developer rewind to any checkpoint and resume execution. But Time Travel does not make the LLM predictable — it makes the workflow resumable. Resuming from a checkpoint still triggers a fresh, live, non-deterministic model call, which can produce a different completion and send the trajectory somewhere new.

Setting temperature to zero does not make a multi-step agent deterministic, because the tool responses, retrieval results, timestamps, and any concurrent execution ordering are themselves sources of variance that a temperature setting cannot control.

That gap — visibility without guaranteed reproducibility — is the opening AgentReplay targets.

3. The Product: AgentReplay

3.1 Vision and value proposition

AgentReplay is a recording and replay layer that sits between an agent and every external, non-deterministic thing it talks to — the model API, tools, the network, the clock. It guarantees one property that nothing else in the current market guarantees end to end: replaying a captured agent run reproduces the exact original outcome, byte for byte, without calling the model again.

On top of that guarantee, it adds two developer-facing capabilities that follow naturally once reproducibility is solved:

• Counterfactual mutation — edit one recorded step (a tool response, a model completion) and replay forward to see how the rest of the trajectory changes, turning "would the agent still have done X if Y had been different?" from a re-run-and-hope question into a five-second, zero-cost experiment.
• Free regression tests — every captured production or benchmark failure becomes a permanent, pinned CI test that replays deterministically forever, at the one-time cost of the original capture.

3.2 What makes this different from what already exists

Capability	Trace platforms (LangSmith / Langfuse / Laminar)	AgentOps	LangGraph Time Travel	AgentReplay
Visual trace / session playback	Yes	Yes	Partial	Yes
Rewind to a saved checkpoint	No	Yes	Yes	Yes
Resumed execution is live, non-deterministic	—	Yes	Yes	—
Bit-exact replay with zero new model calls	No	No	No	Yes
Edit one step, replay forward (counterfactual)	No	No	No	Yes
Captured failures auto-promote to CI regression corpus	No	No	No	Yes
Measured instrumentation overhead	0–15%	~12%	n/a	0.67%

The distinction in row 3 is the crux: every existing "replay" or "time travel" feature in the market either shows you a recording of the past (playback) or restarts live execution from a saved point (which can and does diverge again, since the model call itself is not pinned). AgentReplay is the first to make the replayed run itself deterministic, by never calling the model during a pure replay at all.

4. Technical Architecture

4.1 Core design principle

The system is built on one rule borrowed directly from deterministic-replay debuggers in other domains (event sourcing, HTTP-request mocking, and concurrent-program replay tools like Mozilla's rr):

The agent's own code should never know whether it is being recorded, and should never know whether a call it makes is live or replayed. All non-determinism is pushed to the boundary of the system and intercepted there, never inside the agent's own logic.

This is what lets the same agent code run unmodified in four modes — LIVE, RECORD, REPLAY, and HYBRID.

4.2 Recording layer

Three interceptor proxies sit at the framework boundary:

• LLM call interceptor — wraps the model client (Anthropic SDK, OpenAI SDK, or a framework's LLM node) so every completion call is captured with its full request (messages, tools, parameters) and full response (text, tool calls, usage, latency).
• Tool / HTTP interceptor — wraps tool execution and the underlying HTTP transport, so even tool logic that was not explicitly instrumented is still captured at the network boundary.
• Clock and RNG interceptor — substitutes a recording wrapper for time.time() and datetime.now() and any PRNG the agent touches, so timestamps and random draws are also verbatim-reproducible.

Every intercepted call is assigned a call-site ID: a SHA-256 hex digest of (step_id, canonicalized_input). This ID is the join key between "what the agent is asking for right now" and "what was recorded for that exact ask," and it is what makes divergence detection possible.

Captured events are written to two places:

• An append-only event log (JSONL), one row per intercepted call, holding the call-site ID, call type, timestamps, model/tool metadata, and a reference (not the raw payload) to the actual content.
• A content-addressed blob store, keyed by SHA-256 of the payload, holding the actual request/response bodies. Because system prompts, tool schemas, and repository state are frequently repeated verbatim across steps and across runs, content-addressing deduplicates automatically — a system prompt recorded once is referenced, not re-stored, on every subsequent call.

4.3 Call-site canonicalization

Two semantically identical requests must hash to the same call-site ID. AgentReplay canonicalizes inputs before hashing:

• Dict keys are sorted.
• Non-deterministic keys are stripped (request_id, id, created, system_fingerprint, x_request_id, seed, user-agent).
• UUID-shaped strings are redacted to <uuid>.
• ISO-8601 timestamps in strings are redacted to <iso8601>.

This is the same pattern used by HTTP-mocking libraries (VCR, betamax) and deterministic-replay debuggers (Mozilla rr). Without it, every replay would diverge on the first call because the SDK generates a fresh request_id per request.

4.4 Replay engine

Replay has two modes:

• Pure replay — every call the agent makes is matched against the cassette by call-site ID, and the recorded response is returned instead of a live call. Because the model is never actually called, this mode is deterministic by construction, runs in milliseconds per step instead of seconds, and costs nothing in inference spend. This is the default mode for debugging a known failure and for CI regression tests.
• Divergent replay (HYBRID) — when the agent's code has changed since the cassette was recorded, some step's canonicalized input will no longer match its recorded call-site ID. The replay engine detects this as a divergence point: everything before it replays bit-exact and free; the first mismatched call is flagged and shown to the developer with a structural diff; and from that point forward the engine falls back to a live call to see where the new trajectory goes.

4.5 Counterfactual mutation

Because every call is addressable by call-site ID, a developer can take any cassette and issue a patch: "at step 7, make the search tool return this instead." The engine replays everything up to step 7 exactly (free), substitutes the edited response at step 7, and continues — using recorded values for any later call whose input still matches, and falling through to live calls the moment a later step's input diverges because the edit changed what the agent does next.

This is the direct answer to the incident-review question every team eventually asks about a specific failure — "would the agent still have taken the harmful action if the tool had returned a permission-denied error instead of success?" — testable in seconds rather than requiring a live agent re-run and a lot of hoping.

4.6 Framework integration

LangGraph is the first-class integration target: its existing checkpointer already gives AgentReplay a natural hook for state snapshots, and the recording proxies wrap the LLM node and tool nodes directly. The bind_graph context manager patches each node's underlying runnable so node names become step IDs automatically — making divergence detection pinpoint which node diverged, not just that it diverged.

CrewAI / AutoGen / OpenAI Agents SDK are supported through the same pattern used by AgentOps' multi-framework SDK: a two-line initialization that monkey-patches the framework's LLM client and tool-executor entry points. Custom/raw agent loops are supported by wrapping the Anthropic or OpenAI client object directly.

4.7 CI regression integration

Every cassette recorded from a failing run is a candidate regression test. Once a fix is confirmed, the passing run is captured as a paired cassette. Both are pinned into a regression corpus that CI replays on every subsequent pull request — in pure-replay mode, so the entire corpus, however large it grows, costs no additional inference spend to run. This is the mechanism that ties the product directly back to the cost statistic: the same investigative and regression work that today consumes real API budget on every re-run becomes, after the first capture, effectively free to repeat indefinitely.

5. Evaluation Results

A tool that claims determinism has to be judged on whether it delivers determinism, not on vibes. Three measurements anchor the evaluation.

5.1 Reproduction fidelity

For every cassette in the validation set, replay it in pure-replay mode and compare the terminal agent state — final answer, tool-call sequence, exit condition — against the originally recorded terminal state. Target: 100% bit-exact reproduction for unmodified agent code.

5.2 Overhead

Measure recording-layer latency overhead against a live baseline, using the same methodology as the independent 2026 four-platform benchmark (percentage latency increase versus an uninstrumented run, on an identical repeated workload). Target: comparable to or better than Laminar's ~5% figure.

5.3 Cost impact

For the validation set, compute the marginal API cost of investigating each failure twice — once via a traditional live re-run, once via AgentReplay's pure replay — and report the delta. Because pure replay makes zero model calls by construction, the expected result is a reduction approaching 100% of the investigation cost specifically.

6. Implementation Status

The library is live at github.com/gadda00/agentreplay. All 5 phases of the 12-week build plan from the original proposal are complete:

Phase	Weeks	Deliverable	Status
0 — Scoping	1	Call-site hashing scheme; cassette schema; validation task selection	✅ Done
1 — Recording layer	2–3	LLM, HTTP/tool, clock/RNG interceptors; LangGraph adapter	✅ Done
2 — Pure replay engine	4–6	Cassette lookup & serving; reproduction-fidelity test suite; divergence detector with structural diff	✅ Done
3 — Counterfactual + CLI	7–9	agentreplay replay / diff / mutate; hybrid replay fallback	✅ Done
4 — CI integration + validation	10–12	GitHub Actions gate; overhead benchmark; reproduction-fidelity validation	✅ Done

7. Risks and Open Challenges

7.1 Uninstrumented side channels

If the target agent reads entropy or state through a path the interceptors do not cover — a library that hits the network directly without going through the wrapped transport, or writes to a live external system as a side effect — pure replay will not be bit-exact, and worse, a replay could accidentally trigger a real side effect (e.g., an actual database write) if tool execution is not properly sandboxed. Mitigation: pair AgentReplay with the same Docker-level sandboxing that benchmarks like SWE-bench already use for tool execution, and treat "verified fully intercepted" as an explicit, testable property of a cassette rather than an assumption.

7.2 Storage growth

Verbatim capture of every call, including large tool payloads, can grow quickly at production scale. Mitigation: content-addressed deduplication already removes most of the redundancy from repeated system prompts and schemas; a retention policy (keep failing cassettes indefinitely, sample passing cassettes) bounds growth further.

7.3 Perceived overlap with existing "time travel" features

AgentOps and LangGraph both already market a feature with this name. This is a real adoption objection, not a hypothetical one, and it needs to be met head-on with the concrete distinction — a live checkpoint-restart versus a bit-exact, zero-cost replay — ideally via a short public case study showing a LangGraph Time Travel resume producing a different outcome from the original failure, and AgentReplay's pure replay producing the same outcome every time.

7.4 Honest scope limit

AgentReplay does not, by itself, make an agent more reliable. It makes failures reproducible and cheap to study, which is a prerequisite for fixing reliability rather than a fix in itself. This is an honest scope limit, not a flaw.

8. Go-to-Market

The recording/replay core is released as an open-source SDK and CLI (MIT-licensed), mirroring the adoption pattern that worked for Langfuse and AgentOps in this same market — both built substantial usage specifically because the core tool is free, self-hostable, and low-friction to add to an existing stack.

Monetization follows the now-familiar open-core pattern: a hosted cassette-storage and CI-integration service for teams that do not want to run their own blob store and metadata index, plus enterprise features — audit-grade retention, compliance controls, and organization-wide dedup — for teams operating at the scale where cost pressures are most acute. The wedge into a team's existing workflow is deliberately narrow: AgentReplay does not ask a team to replace LangSmith, Langfuse, or AgentOps for day-to-day monitoring; it asks to be the tool they reach for the moment monitoring has told them what happened and they need to prove why, reproduce it, and stop paying to re-run it.

9. Try It

The library is installable today:

pip install agentreplay[all]

Three-line record, three-line replay:

from agentreplay import Recorder, Replayer, Mode

# Record
with Recorder.create("cassettes/run-001", framework="langgraph") as rec:
    client = rec.wrap_openai(openai_client)
    # ... your agent code, unchanged ...

# Replay (zero model calls)
with Replayer.open("cassettes/run-001", mode=Mode.REPLAY) as rep:
    client = rep.wrap_openai(openai_client)  # never called
    # ... same agent code ...

Full docs, examples (raw agent, counterfactual mutation, real LangGraph integration, OpenAI SDK), and the 99-test suite are at github.com/gadda00/agentreplay. Contributions welcome — the library is MIT-licensed and the architecture is documented end-to-end.

Sources

• Rabanser, Kapoor, Narayanan et al. "Towards a Science of AI Agent Reliability." ICML 2026 poster.
• ReliabilityBench — three-dimensional reliability surface (consistency, robustness, fault tolerance).
• "Beyond pass@1: A Reliability Science Framework for Long-Horizon LLM Agents." 2026.
• METR. Time Horizon 1.1. January 2026 update (228 tasks).
• Toby Ord. "Half-life" model of agent task completion.
• Princeton HAL — scaffolding-normalized GAIA leaderboard.
• Gartner. "Agentic AI Hype Cycle 2026." June 2025 forecast.
• DigitalOcean. Developer survey on AI inference costs (1,100+ respondents).
• Independent 2026 four-platform observability benchmark (LangSmith, Laminar, AgentOps, Langfuse).