Backtesting Engine

A vectorized backtest computes signals and returns over whole arrays at once. It's quick to write and fast to run, and it's easy to get wrong. One misplaced shift and the signal at row t uses row t + 1. It also can't express queue position, partial fills, latency, or a cancel that arrives too late.

An event-driven backtest runs the same loop production does:

recorded data ──► event queue (ordered by delivery time, ties by sequence)
                     │
                     ├─► Md(update)      → strategy.on_market_data   (after md latency)
                     ├─► OrderArrive(o)  → engine.submit(o)          (after order latency)
                     └─► Ack/Fill(o)     → strategy.on_exec, portfolio (after ack latency)
strategy decisions ──► send_order(now) ──► scheduled OrderArrive

Two properties are worth the extra effort:

• The strategy can't see the future, because it only ever receives events whose delivery time has passed. The order of the loop enforces this, not anyone's carefulness.
• The strategy under test is the production strategy, behind the same interface and taking the same callbacks. Whatever you validate in the backtest is the code you'll run.

Determinism carries over from week 9. The same recorded input, latency model and seed must produce byte-identical outputs, so equal timestamps need a defined tie-break. A backtester that gives a different PnL on every run can't be debugged, and it can't detect regressions either.

Here's a short backtest with three bugs. Find them before reading on.

// A "momentum" backtest. Compiles, runs, looks great.
let n = prices.len();
let mean = prices.iter().sum::<i64>() as f64 / n as f64;           // (a)
let mut pnl = 0i64;
for t in 1..n - 1 {
    let ma = (prices[t - 1] + prices[t] + prices[t + 1]) / 3;        // (b)
    let pos = if prices[t] as f64 > mean && prices[t] > ma { 1 } else { -1 };
    pnl += pos * (prices[t + 1] - prices[t]);                        // (c)
}

(a) The full-sample mean includes every future price. (b) A centred moving average includes prices[t + 1]. (c) is subtler: the position decided from prices[t] is assumed to be entered at prices[t], the same price that produced the decision. In reality you see that price, decide, and your order arrives later, after the market has moved on. (a) and (b) leak information. (c) leaks time.

The cheapest defence against information leaks is a perturbation test. Change every input after index k, rerun, and assert that nothing at or before k changed. It catches (a) and (b) mechanically. It doesn't catch (c), because nothing at (c) uses a future value. It just pretends execution takes no time. That's what the explicit execution delay in today's drill, and tomorrow's latency model, are for.

Related biases to watch for:

• Timestamp choice. Decide on the time you would have received the data (exchange timestamp plus feed latency), not the time the exchange stamped it.
• Revised data. Reference data, corporate actions and index membership as known at the time, not as corrected later ("as-of" lookups).
• Survivorship. A universe built from today's instrument list (week 43).

/// Trend signal for each index t, using ONLY prices[..=t]:
///   +1 if prices[t] is above the mean of the `window` prices BEFORE t
///      (prices[t − window .. t], which excludes t itself),
///   −1 if below, 0 if equal or if t < window.
/// Compare in integers: prices[t] · window vs the sum. No floats needed.
pub fn signals(prices: &[i64], window: usize) -> Vec<i8> {
    todo!()
}

/// Equity curve of trading 1 lot toward the signal, with an execution
/// delay of `delay` steps (the decision made at t fills at prices[t + delay]).
/// Start flat with 0 cash. For each step t in 0..n:
///   1. if t ≥ delay: trade to position signals[t − delay] at prices[t]
///      (buying costs price·qty cash, selling earns it);
///   2. equity[t] = cash + position · prices[t].
/// Decisions whose fill step would be ≥ n never execute.
pub fn equity_curve(prices: &[i64], signals: &[i8], delay: usize) -> Vec<i64> {
    todo!()
}

Model at least three delays, and then your own decision time:

• Market data: exchange → strategy. You react to a book that's already this old.
• Order entry: strategy → exchange. Your order joins the queue behind everyone who got there in the meantime, and your cancels take this long too. Many of the fills that hurt a market maker happen on quotes it was already trying to cancel.
• Acks and fills: exchange → strategy. You learn about your fills late, so your position is uncertain for that window.
• Decision time: the strategy's own processing. Charge it explicitly, or a slow model will look as fast as a lookup table.

Use distributions rather than constants once the basics work, but keep one physical constraint: orders on a single session arrive in the order they were sent. TCP doesn't reorder, so a jitter model that lets order 2 overtake order 1 is simulating something that can't happen.

The engine of it all is a priority queue keyed by (delivery_time, sequence). The sequence number makes ties deterministic: two events due at the same nanosecond pop in the order they were scheduled, every run.

use std::cmp::Reverse;
use std::collections::BinaryHeap;

#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Ev {
    /// Market data update `id` reaches the strategy.
    Md(u32),
    /// Our order `id` reaches the exchange.
    OrderArrive(u32),
    /// The exchange's acknowledgement of order `id` reaches the strategy.
    Ack(u32),
}

/// A discrete-event clock for a backtest: every event has a delivery time
/// in simulated ns, and `next` hands them out in time order.
pub struct LatencySim {
    md_ns: u64,
    order_ns: u64,
    ack_ns: u64,
    // Suggested: a min-heap keyed by (time, insertion seq) so equal times
    // pop in the order they were scheduled — determinism depends on it.
    heap: BinaryHeap<Reverse<(u64, u64, u32, u8)>>,
    seq: u64,
    last_order_arrival: u64,
}

impl LatencySim {
    pub fn new(md_ns: u64, order_ns: u64, ack_ns: u64) -> Self {
        todo!()
    }

    /// The exchange published update `id` at exchange time `ts`; the
    /// strategy sees it md_ns later.
    pub fn publish_md(&mut self, ts: u64, id: u32) {
        todo!()
    }

    /// The strategy sends order `id` at `now`. It reaches the exchange at
    /// now + order_ns + extra_ns (extra_ns models jitter) — but never
    /// before an order sent earlier: one session is one TCP stream, and
    /// TCP delivers in order.
    pub fn send_order(&mut self, now: u64, id: u32, extra_ns: u64) {
        todo!()
    }

    /// Pop the earliest event; ties pop in scheduling order. Popping
    /// OrderArrive(id) at time t means the exchange accepted it, so it
    /// also schedules Ack(id) for t + ack_ns.
    pub fn next(&mut self) -> Option<(u64, Ev)> {
        todo!()
    }
}

Aggressive orders are the easy case. At the order's arrival time, walk the book: fill against resting liquidity level by level at the makers' prices, up to your limit. Use the book at arrival, not at decision time. The difference between the two is the latency cost you're trying to measure.

Passive orders are where backtests flatter themselves. Three common models, from optimistic to conservative:

1. Touch fill. Filled as soon as the market trades at your price. Optimistic, because there may have been a long queue ahead of you that absorbed all of that volume.
2. Queue model. Track the size ahead of you from the moment your order arrives (week 41), reduce it by trades and an assumed share of cancels, and fill only once it's used up. A reasonable middle ground. How optimistic it is depends on the cancel assumption.
3. Trade-through fill. Filled only when the market trades at a price worse than yours, which proves your level was fully consumed. Conservative. It misses some fills that would really have happened.

Replaying an order-level recording through your own engine, with your orders inserted, makes the queue model concrete: your order gets an actual place in the FIFO behind real recorded orders. It still doesn't solve the counterfactual problem. If your order takes liquidity, a recorded trade that later tried to hit the same orders finds them gone, and from then on the replayed book differs from history. The recorded participants never react to you either. Pick a rule for divergence: skip recorded executions against liquidity you consumed, or cap your size at a fraction of displayed liquidity so divergence stays small. Document the rule, and report how often it applied.

Then add the parts everyone forgets: fees and rebates by order type, tick and lot rounding, partial fills, and rejects from risk checks and throttles (weeks 15 and 17).

A backtester needs tests of its own, and the useful ones target optimism:

• Do nothing. A strategy that never trades has exactly zero PnL and zero fills.
• Pay to play. A strategy that trades at random loses roughly its costs: spread plus fees, times turnover. If it makes money, something is free that shouldn't be.
• Latency is never free. PnL at realistic latency ≤ PnL at zero latency, within noise, for the strategies you care about.
• Information never travels back. Perturbation tests (today's drill) on every signal and on the equity curve.
• Determinism. Hash the full output (every fill and PnL point). Two runs give the same hash, and so does a run on another machine.
• Conservation. Every fill the strategy receives matches an engine trade, quantities add up, and no order fills more than its size.
• Reconciliation. Once you have any live trading, compare live fills with what the backtester predicts for the same period. The gap ("slippage versus backtest") is the most honest measure of backtester quality there is.

How fast the replay runs decides how many experiments you can run. So it's an engineering target, and also, as week 43 showed, a risk: more experiments, more overfitting.

At the replay-speed target below (4 million messages a second), each message gets about a quarter of a microsecond for everything: decode, engine update, event-queue push and pop, strategy callback and accounting. Where that time goes:

• Input. Read recordings in large sequential chunks or memory-map them, and decode zero-copy (week 11). Never allocate per message.
• Event queue. A binary heap is O(log n) per event, which is fine when the queue is short. Most events are market data already in time order, so merge the recorded stream with a small heap of simulated events (orders and acks) instead of pushing every recorded message through the heap.
• Engine and strategy. Already allocation-free from Phase 3.
• Parallelism across replays, not within one. Different days, instruments and parameter sets are independent. Run one per core. Splitting a single replay across threads would cost you its deterministic ordering, which is the property the whole backtester depends on.