Chapter 10 of 10 · 9 entries

10. Backtesting, Replay, and Leakage Control

Finance-specific reliability layer: point-in-time data, look-ahead bias control, deterministic replay, and realistic cost models. General agent durability belongs in agent-infra style lists; this chapter is market-time correctness.

Harnesses and engines

• TraderHarness ⭐ Contamination-resistant point-in-time backtest and replay for LLM traders.
• nautilus_trader Deterministic production-grade trading engine usable as an agent execution substrate.
• vectorbt High-performance vectorized backtesting often wrapped by strategy agents.
• backtesting.py Simple, readable Python backtesting library for strategy prototypes.
• pybroker Python algorithmic trading with ML-friendly backtesting hooks.
• zipline-reloaded Maintained Zipline fork for Pythonic event-driven backtests.
• Lean QuantConnect's open-source event-driven engine for backtests and live trading.
• llm-agent-trader Backtesting system where an LLM makes the trade decisions, with FastAPI and Next.js UI.
• vectorbt-backtesting-skills Agent skills for writing and running VectorBT backtests (India, US, crypto).

Pitfall Tip

If your backtest can see tomorrow's close, your agent is cheating. Prefer PIT datasets, embargoed splits, and replay logs that pin model + data versions.