Simulation & Integration Testing

A simulation is deterministic only if every input that can vary between runs is either fixed or derived from the seed. Here is the list, roughly in order of how often each one bites:

• Randomness. Every random choice comes from a seeded PRNG. Give each simulated agent its own generator, seeded from (global seed, agent id). Then adding a new agent doesn't change every existing agent's stream, and a scenario stays recognizable as it evolves.
• Time. Nothing reads a real clock. The simulation owns a virtual clock (a manual clock like the one sketched in week 21) and advances it to the next scheduled event. A simulated hour can run in a second. The repo's runner goes one step simpler: its now is the step count.
• Scheduling. Real threads interleave differently every run. In simulation mode, run every component (agents, gateway sessions, engine, publisher) on one thread, stepped by a scheduler in a fixed order, with in-memory transports instead of sockets.
• Hash iteration order. Rust's HashMap uses a randomly keyed hasher (RandomState) that differs per process, so iterating one gives a different order every run. Anywhere iteration order can leak into output (which session gets its events first, which order is cancelled in a mass cancel), use a BTreeMap, a Vec, or a fixed hasher.
• Addresses and ids. Pointer values, thread ids and anything derived from the OS must never become keys or tie-breakers.

This is the approach FoundationDB made famous: the entire distributed database runs in a deterministic simulator with simulated network, disks and failures, and any failing seed reproduces exactly. TigerBeetle does the same with its VOPR simulator. You need far less than they built, because your exchange is already single-threaded where it matters.

Keep two modes and share as much as possible between them:

1. Simulation mode: one thread, virtual time, in-memory transport. Used for correctness: scenarios, invariants, seed sweeps, replay of failures.
2. Load mode: real threads, real sockets on loopback, real clocks. Used for performance and concurrency (week 24). The same agents and scenarios run here, but you assert outcomes, not orderings.

The default model for "orders arrive at random" is a Poisson process: arrivals at a constant average rate λ, each independent of the last. Two facts make it the right starting point. It's memoryless: the time until the next arrival doesn't depend on how long you've already waited. And the superposition of many independent senders tends towards a Poisson process (the Palm–Khintchine theorem), which is roughly what a venue's aggregate flow looks like over short windows.

The gaps between arrivals are exponentially distributed with mean 1/λ, and you draw them by inverse transform: if u is uniform on [0, 1), then −ln(1 − u) / λ is exponential. (1 − u rather than u because a uniform generator can return exactly 0, and ln(0) is −∞: one infinite gap and your simulation stops producing orders.)

Real flow is not that tidy, and it's worth knowing how:

• Intraday seasonality. Volume is typically heaviest around the open and the close. You can model a time-varying rate λ(t) by thinning: generate candidates at the maximum rate and keep each one with probability λ(t)/λ_max (Lewis and Shedler's method).
• Clustering. Orders cause orders: a trade triggers quote updates, which trigger more trades. Self-exciting (Hawkes) processes capture this, and plain Poisson underestimates bursts.
• Cancellations dominate. Most orders on modern electronic venues are cancelled rather than filled. A flow model without heavy cancel traffic doesn't exercise the paths that real traffic hammers.

For testing an exchange, coverage matters more than realism. You want flow that crosses the spread, sweeps several levels, empties one side of the book, cancels orders a microsecond after they filled, and hits every reject path. The academic "zero-intelligence" models are a good base precisely because they're simple. Random limit orders, market orders and cancels at Poisson rates already reproduce realistic-looking spreads and depth.

/// SplitMix64 (given): tiny, fast, good enough for simulation, and seeded.
pub struct Rng(u64);

impl Rng {
    pub fn new(seed: u64) -> Self {
        Rng(seed)
    }

    pub fn next_u64(&mut self) -> u64 {
        self.0 = self.0.wrapping_add(0x9E37_79B9_7F4A_7C15);
        let mut z = self.0;
        z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9);
        z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB);
        z ^ (z >> 31)
    }

    /// Uniform in [0, 1) with 53 random bits. CAN return exactly 0.0.
    pub fn next_f64(&mut self) -> f64 {
        (self.next_u64() >> 11) as f64 / (1u64 << 53) as f64
    }
}

/// Inverse-transform sample of an exponential gap, in seconds, for an
/// arrival rate of `rate` per second: −ln(1 − u) / rate. Finite for u = 0.
pub fn exp_from_uniform(u: f64, rate: f64) -> f64 {
    let _ = (u, rate);
    todo!()
}

/// Arrival times (ns) of a Poisson process with `rate` arrivals per second
/// on [0, horizon_ns). Each gap is rounded to whole ns and is at least
/// 1 ns, so timestamps strictly increase. Same seed → same output.
pub fn arrivals(seed: u64, rate: f64, horizon_ns: u64) -> Vec<u64> {
    let _ = (seed, rate, horizon_ns);
    todo!()
}

/// Time-varying rate by thinning (Lewis–Shedler): generate candidates at
/// `max_rate` exactly like `arrivals`, and keep a candidate at time t with
/// probability rate_at(t) / max_rate (draw one more uniform per candidate).
pub fn arrivals_varying(seed: u64, max_rate: f64, rate_at: &dyn Fn(u64) -> f64, horizon_ns: u64) -> Vec<u64> {
    let _ = (seed, max_rate, rate_at, horizon_ns);
    todo!()
}

#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Kind {
    Limit,
    Market,
    Cancel,
}

/// `n` order kinds drawn with relative weights [limit, market, cancel].
/// A zero weight must never be drawn.
pub fn order_kinds(seed: u64, n: usize, weights: [u32; 3]) -> Vec<Kind> {
    let _ = (seed, n, weights);
    todo!()
}

Give every simulated participant the same small interface. It reacts to what its session receives and to timers it scheduled, and it acts through a context that owns the RNG, the clock and the outbound queue:

pub trait Agent {
    fn on_start(&mut self, ctx: &mut Ctx);
    /// Acks, fills, rejects and market-data updates for this agent's session.
    fn on_event(&mut self, ev: &ClientEvent, ctx: &mut Ctx);
    /// A wake-up the agent asked for with ctx.schedule(at).
    fn on_timer(&mut self, now: u64, ctx: &mut Ctx);
}

pub struct Ctx<'a> {
    pub now: u64,
    pub rng: &'a mut Rng,          // this agent's own seeded stream
    out: &'a mut Vec<ClientMsg>,   // drained into the agent's gateway session
    timers: &'a mut TimerQueue,
}

A cast that covers the paths you care about:

• Market makers quote both sides around a reference price, cancel and replace when filled or when the reference moves, and respect an inventory limit. They are what makes the book look like a book.
• Takers send IOC or market orders at Poisson times, sometimes large enough to sweep several levels.
• Noise traders place limit orders at random distances from the touch and cancel them at random. This is your cancel traffic.
• Troublemakers exercise the defensive code: a fat-finger agent that occasionally sends absurd prices and sizes (week 15 risk), a burst agent that exceeds its throttle (week 17), a slow market-data subscriber (week 16), and a reconnector that drops its session mid-stream and recovers with sequence numbers (week 14).

Drive the reference price with a simple random walk, so makers move and takers chase. The goal isn't a realistic market. It's a busy one that touches every code path, every run, from a seed.

Random flow finds the bugs you didn't think of. Scripted scenarios pin down the ones you did: partial fill then cancel, a cancel racing a fill, a sweep through three levels, a modify-down that keeps priority. They should read like specifications, so that someone who knows market structure but not Rust can review them.

The one design decision that matters most in the runner is strictness: every output must be expected. A runner that only checks what the script mentions will pass a script that forgot about a fill, which is exactly the bug class (an extra fill, a duplicate ack) you most need to catch. In the drill, an unconsumed fill fails the scenario and is blamed on the submit that produced it.

#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Side {
    Buy,
    Sell,
}

#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Fill {
    pub maker: u64,
    pub taker: u64,
    pub qty: u64,
    pub price: u64,
}

/// What a scenario drives. In the repo: the whole exchange behind the
/// gateway. Here: whatever the harness plugs in (including buggy ones).
pub trait Venue {
    /// Submit a limit order; returns the fills it caused, in match order.
    fn submit(&mut self, id: u64, side: Side, qty: u64, price: u64) -> Vec<Fill>;
    /// Cancel a resting order; false if it was not resting.
    fn cancel(&mut self, id: u64) -> bool;
    /// Total resting quantity at one price level.
    fn level_qty(&self, side: Side, price: u64) -> u64;
}

#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Failure {
    Parse(String),
    Mismatch { expected: String, got: String },
    UnexpectedFills(usize),
}

#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ScenarioError {
    /// 1-based line number in the script.
    pub line: usize,
    pub failure: Failure,
}

/// One command per line; tokens are separated by whitespace; blank lines
/// and lines starting with '#' are skipped.
///
///   buy <id> <qty> @ <price>
///   sell <id> <qty> @ <price>
///   cancel <id>
///   expect fill <maker> <taker> <qty> @ <price>
///       pop the next unconsumed fill of the latest submit and compare
///   expect no-fills
///       the latest submit has no unconsumed fills left
///   expect level <buy|sell> <price> <qty>
///       resting quantity at that level
///   expect cancel-rejected
///       the latest cancel returned false
///
/// Anything else (unknown command, wrong token count, bad number, missing
/// '@') is Failure::Parse at that line. A failed expectation is
/// Failure::Mismatch at that line.
///
/// Strictness: every fill must be expected. If a buy/sell/cancel line is
/// reached, or the script ends, while fills from the latest submit are
/// still unconsumed, fail with UnexpectedFills(how many) reported at the
/// line of the SUBMIT that produced them.
///
/// Returns the number of `expect` lines checked.
pub fn run_scenario<V: Venue>(venue: &mut V, script: &str) -> Result<usize, ScenarioError> {
    let _ = (venue, script);
    todo!()
}

Scenarios check specific cases. Invariants check every event of every run, scripted or random. In simulation mode you can afford to check them after each event:

• Conservation. For every order, original quantity = filled + resting + cancelled. For every trade, the buy and sell quantities match.
• Market data agrees with the engine. After applying market-data sequence N, the subscriber's book equals the engine's book as of the command that produced N.
• Per-session ordering. An ack precedes any fill of that order, nothing follows a terminal state, and a cancel ack is never followed by a fill.
• Risk holds. No account's position or open exposure ever exceeds its configured limits (week 15).
• Liveness. At the end of a run, every order is resting or has reached a terminal state. Nothing is stuck in flight.

Finally, a golden hash: hash the full output event stream of a fixed-seed run and commit it. Any change to that hash is either a bug or an intended behaviour change, and either way a human should look at it (the insta crate makes this kind of snapshot review pleasant).

When a simulation fails, the workflow should be mechanical:

1. The failure prints its seed (and the scenario name), and rerunning with that seed reproduces it exactly.
2. Shrink it with the week 12 machinery: fewer agents, a shorter run, fewer orders, until the failure is small enough to read.
3. Replay the journal. The journal of the failing run is a complete recording of the inputs. Replay it into a fresh engine with tracing turned on and step to the interesting sequence number.
4. Bisect divergence. When two runs that should match don't (before and after a refactor, primary and standby), compare the state hash at each sequence number and binary-search for the first one that differs. That command is your bug.

Load mode needs a different discipline. With real threads, the interleaving of events from different sessions legitimately varies between runs. Assert what must hold regardless of interleaving: per-session ordering, conservation, final book state, the invariants above. Never assert a global order of events across sessions. A test that does is flaky by construction.

Build the simulation crate below, then run a seed sweep (a thousand seeds of a few simulated seconds each is a reasonable CI job) and fix whatever falls out. Expect to find at least one real bug in an earlier week's code. That's the point.