Statistical Arbitrage Basics

Prices behave roughly like random walks. They're non-stationary: they drift and don't return to any fixed level. That has two consequences:

• Regressing one price on another can mislead you. Two independent random walks, regressed on each other, often give a "significant" slope and a high R². This is the spurious regression problem (Granger and Newbold, 1974). A good fit between two price series on its own tells you almost nothing.
• Correlation of returns isn't enough. Two stocks whose daily returns are 0.9 correlated can still drift apart without limit over months. Correlation describes how they co-move from day to day. It says nothing about whether the gap between their levels stays bounded.

What a mean-reversion trade needs is cointegration (Engle and Granger, 1987): two non-stationary series x and y such that some combination y − β·x is stationary. That stationary spread is what you trade. The Engle–Granger two-step check regresses y on x in levels, then tests the residual for a unit root with an augmented Dickey–Fuller test. Use critical values adjusted for the fact that β was estimated, not the standard Dickey–Fuller tables. The Johansen test generalizes this to more than two series.

Statistics alone won't find pairs that keep working, though. You want an economic reason for the link: the same underlying in two forms (an ETF and its basket, a future and its cash index, two share classes of one company, a dual listing). "Two companies in the same industry" is a much weaker link, and it breaks in exactly the situations that cost the most: a merger, a scandal, a change in strategy.

The textbook hedge ratio is the OLS slope of y on x: β = cov(x, y) / var(x). The spread is then s = y − β·x − α. Three things are worth knowing before you rely on it:

• OLS isn't symmetric. Regressing y on x minimizes vertical errors. Regressing x on y minimizes horizontal ones. With noisy data the two slopes aren't reciprocals: their product is R², not 1. Which instrument you call the dependent variable changes the hedge. Total least squares (orthogonal regression) treats the two symmetrically, at the cost of being less familiar.
• Units matter. β from prices is in units of y per unit of x. Before you size orders, convert to quantities per leg using contract multipliers and lot sizes. Rounding to whole lots leaves a small unhedged remainder that you have to track.
• The relationship drifts. A β fitted once becomes stale. Rolling-window or recursive estimates (a Kalman filter is the common choice) keep up with the drift, but a noisy β adds noise to the spread. Decide the trade-off deliberately.

The half-life of the spread tells you how fast it reverts. Fit Δs_t = c + λ·s_{t−1} by OLS. If λ is negative, the spread is pulled back toward its mean, and the half-life is about −ln 2 / λ steps. It's the natural scale for your z-score window and expected holding time. If the half-life comes out at months, you're not looking at a high-frequency trade.

/// Ordinary least squares fit of y = alpha + beta · x.
/// beta = cov(x, y) / var(x), alpha = mean(y) − beta · mean(x).
/// Returns (alpha, beta). Assumes x.len() == y.len() ≥ 2 and var(x) > 0.
/// Tip: compute the means first, then sum the centered products — the
/// one-pass Σxy − n·x̄·ȳ form cancels badly for prices.
pub fn ols(x: &[f64], y: &[f64]) -> (f64, f64) {
    todo!()
}

/// Residual spread s_t = y_t − beta · x_t − alpha.
pub fn spread(x: &[f64], y: &[f64], alpha: f64, beta: f64) -> Vec<f64> {
    todo!()
}

/// Half-life of mean reversion of a spread, in steps.
/// Regress the change on the previous level:  Δs_t = c + λ · s_{t−1}
/// (Δs_t = s_t − s_{t−1}, t = 1..n). If λ < 0 the spread pulls back
/// toward its mean and the half-life is −ln 2 / λ; otherwise None.
/// Also None with fewer than 3 points or when s_{t−1} has zero variance.
pub fn half_life(s: &[f64]) -> Option<f64> {
    todo!()
}

The usual signal is the spread's z-score over a trailing window: z = (s − mean_N) / std_N. A typical rule set: enter when |z| exceeds an entry threshold (sell the spread when it's high, buy when low), exit when |z| falls back inside an exit threshold, and stop out beyond a much wider stop threshold, because "it will revert eventually" is how pairs traders blow up.

On a tick-by-tick path you can't recompute mean and variance over the window on every update. Keep a ring buffer of the window and update running statistics as one value enters and the oldest leaves. That's O(1) per tick. There's a numerical trap here. The schoolbook formula var = (Σx² − (Σx)²/n)/(n − 1) subtracts two enormous, almost equal numbers. With prices stored as integer ticks near 1e9 and a window that moves only a few ticks, it loses every significant digit and can even go negative. Use a sliding Welford update, or subtract a fixed reference value before accumulating, and recompute from the buffer now and then to stop rounding drift.

/// Rolling mean / standard deviation / z-score over the last `window`
/// values, updated in O(1) per push. Change the fields if you like —
/// the harness only calls the methods.
pub struct RollingZ {
    window: usize,
    buf: Vec<f64>, // ring buffer of the values in the window
    head: usize,   // next slot to overwrite
    count: usize,  // values currently in the window (≤ window)
    mean: f64,
    m2: f64,       // Σ (x − mean)² over the window (Welford)
}

impl RollingZ {
    pub fn new(window: usize) -> Self {
        todo!()
    }

    /// Add `x`; once the window is full, evict the oldest value. O(1).
    /// Returns z = (x − mean) / std of the window INCLUDING x, but only
    /// once the window is full and std > 0; otherwise None.
    ///
    /// Beware: the textbook Σx and Σx² formula loses every significant
    /// digit when values are large and close together (prices around
    /// 1e9 with a spread of a few ticks). Use a sliding Welford update:
    ///   adding x (window not full):
    ///     n += 1; d = x − mean; mean += d / n; m2 += d · (x − mean)
    ///   replacing old with x (window full):
    ///     new_mean = mean + (x − old) / n
    ///     m2 += (x − old) · (x − new_mean + old − mean)
    pub fn push(&mut self, x: f64) -> Option<f64> {
        todo!()
    }

    pub fn mean(&self) -> f64 {
        todo!()
    }

    /// SAMPLE standard deviation (divide by n − 1) of the values in the
    /// window; 0.0 with fewer than 2 values.
    pub fn std(&self) -> f64 {
        todo!()
    }
}

Pick the window from the half-life, a few half-lives long, not by trying windows until the backtest looks good. The choice of whether the current value counts in its own statistics (the drill includes it) is yours to make. Make it consciously and keep it consistent. The only choice that's always wrong is letting future values into the window.

A signal isn't a trade until both legs are on. That raises problems that one-legged strategies don't have:

• Costs come twice, then twice again. Entering with market orders crosses the spread on both legs, and exiting does it again: four spread crossings plus fees per round trip. Run the numbers. Suppose the spread's standard deviation is 3 ticks, you enter at 2σ (6 ticks) and exit at 0.5σ (1.5 ticks). The gross capture is 4.5 ticks when it reverts. With a one-tick quoted spread on each leg, crossing costs half a tick per leg per side, 2 ticks per round trip before fees. A large share of the edge is gone before you count the trades that never revert.
• Legging risk. To save the crossing cost you can rest one leg passively and hedge with the other only when it fills. Between those two moments you hold an unhedged position in one instrument, and the passive leg tends to fill exactly when the market is moving against it (week 41's adverse selection). Decide in advance how long you'll stay unhedged and what you do when time runs out: cross to hedge, or unwind the filled leg.
• Sizing and rounding. β says 0.73 lots of x per lot of y, but you can only trade whole lots. Size the spread position so the rounding error is small relative to it, and track the residual exposure.
• Limits on both legs and on the spread. Gross exposure per instrument, net exposure of the combined position, and a maximum spread position. Shorting has costs too (borrow for equities, margin for futures) that a price-only backtest leaves out.

Each of these mistakes, on its own, can turn a strategy with no edge into a backtest that looks excellent:

• Look-ahead. Hedge ratio, mean and standard deviation estimated on the full sample (Day 2). Pairs chosen because they reverted over the period you then backtest on.
• Survivorship. A universe built from today's instrument list leaves out the companies that were delisted, merged or went bust. Those are disproportionately the pairs that broke, and the ones where you'd have lost money.
• Multiple testing. Try 1,000 pairs × 20 parameter sets and the best of the 20,000 backtests will look excellent by chance alone. Bailey, Borwein, López de Prado and Zhu (2014) show how quickly the out-of-sample performance of the "best" backtest decays as the number of trials grows. The deflated Sharpe ratio (Bailey and López de Prado, 2014) corrects a Sharpe ratio for the number of trials and for non-normal returns. Harvey, Liu and Zhu (2016) make the same argument for academic factor research and propose much stricter significance thresholds.
• Regime change. Relationships break, and when they do the loss comes all at once rather than gradually. Without a stop, the strategy keeps adding to a losing spread because the z-score says it's cheaper than ever.
• Costs. Four spread crossings per round trip, fees, borrow, slippage on size, and latency (week 44).

Defences are mostly about discipline:

• Keep a holdout period you don't look at until the very end, and look at it once.
• Use walk-forward evaluation: fit on a past window, trade the next one, roll forward.
• Count and report every configuration you tried.
• Prefer parameters with a reason behind them (a window from the half-life, thresholds from costs) to parameters chosen because they gave the best PnL.
• Write down what result would make you abandon the idea before you run it.

On real data you never know the true β or whether the pair is still cointegrated. In a simulator you decide. Build the pair so the truth is known: x follows a random walk, and y is β·x plus an Ornstein–Uhlenbeck spread with a half-life you choose. Then check that your estimated β, half-life and z-scores recover what you put in. That's a test of your estimators, and it's the most valuable thing this simulator can give you.

Then break the pair. At some point in the run, change β, or let the spread follow a random walk instead of reverting. A sound strategy notices through its stop and limits, takes a bounded loss and stops trading. A fragile one keeps adding to the position because the z-score says the spread is cheaper than ever.