Detecting the Unseen
Report
SummaryThe question and the answer in plain words

The study

The learning experimentWhat an AI trader learns, with held-out controls. The detector foundationA market where spoofing works, and seven detectors. ResultsSpoofs against honest look-alikes and volatile markets. What kind of teaching works?Examples, a rule, or the law. And how to dodge it. An AI challengerA search program hunts for spoofs the law-based detector misses. What it meansThe answer, its limits and the sources.

Explore

Research LabPick a market situation and see how every detector reacts. Watch one spoofThe wall goes up, the order fills, the wall vanishes.

The paper

Research report Summary: the project in five minutes Analytics explorer References Progress Code on GitHub

Chapter 3 of 7

Results

What happened when detectors that never saw a spoof met one, and every attempt to break the finding.

Results

All detectors here were trained on calm markets. The three tabs show the same detectors facing single-wall spoofs, layered spoofs, and genuine large orders of the same sizes.

Figure 2 Who catches what

Figure 2. Share of episodes that raised at least one alert, by the size of the large order (multiples of a typical 3-lot order). Colour is the model family, circles look at one bar and squares at a window of 8, dashed lines use the generic features. Bars show 95% confidence intervals; the dotted line is chance. Click a legend entry to pick out one detector.
Show the data for Figure 2

The rule is excellent on the pattern it was written for: of single-wall spoofs from 8× upwards, with almost no genuine orders flagged. Its threshold is lots, so when the same lots arrive as four smaller orders, each piece slips underneath. The untaught detectors never rise far above chance, and the neural network does no better than the simpler models. PCA with spoof-informed features catches the biggest spoofs, but it flags of genuine 16× orders too.

: the rule's catch rate once the same spoof was split into four smaller orders. Every rule invites the next variant.

Can a detector tell a spoof from an honest order?

This is the sharpest test in the project. Take spoofs and genuine large orders of exactly the same size, and ask each detector which looks more unusual.

Figure 3 Spoof or honest order?

Figure 3. AUC for separating spoofs from genuine large orders of the same size. Above 0.5, the detector finds spoofs more unusual. Below 0.5, it finds the honest orders more unusual. Bars show 95% bootstrap intervals.
Show the data for Figure 3

Every untaught detector sits at or below 0.5. The LSTM autoencoder reaches at 16×: it finds the honest orders far stranger than the spoofs. The reason is simple once you see it. An honest large order stays on the book for hundreds of steps, a long run of unusual depth. A spoof is pulled after a handful of steps. The spoof's speed is its camouflage. The only detector that sees through it is the rule, because it was told that being quick is the suspicious part.

Does reading the order of events help?

The sequence detectors were meant to see the pattern: wall appears, opposite order fills, wall vanishes. They did not. Sequence PCA caught significantly fewer large spoofs than plain PCA ( against , ), and the LSTM matched the simpler models. To find out why, every window was scored again with its bars shuffled into a random order.

Figure 4 The order test

Figure 4. AUC of the two sequence detectors with each window's bars in their real order (hollow) and shuffled (filled), at 16×. If a detector used the order of events, shuffling would change its score.
Show the data for Figure 4

Shuffling changes almost nothing, so neither model is using the order of events: they have learned which bars are unusual, not which sequences are. There is a likely reason. A spoof lasts 6 to 10 steps, but each bar summarises 5 steps, so the whole story happens inside one or two bars. The robustness section tests that directly.

Which feature does the teaching?

Removing the spoof-informed features from PCA one at a time shows where its knowledge comes from.

Without the "largest single order" feature, PCA stops flagging most honest orders ( down to ) and starts telling spoofs from them (AUC ), because what is left is mainly "largest order cancelled quickly". That is the rule's definition. Every route to a working detector here runs through that one piece of taught knowledge.

When the market changes

Figure 5 False alarms in a changed market

Figure 5. Share of bars in normal sessions (no spoofing at all) that raised an alert. Hollow: calm test market. Filled: volatile test market. The target is 1%.
Show the data for Figure 5

Does the spoofing actually work?

Figure 6 What the wall achieves

Figure 6. Extra chance that the spoofer's genuine order fills within 30 steps, compared with its honest twin (same market, no wall), in percentage points. Shaded bands are 95% confidence intervals.
Show the data for Figure 6

From 8× upwards, both single-wall and layered spoofs make the genuine order noticeably more likely to fill. So the layered spoofs that every detector missed are not harmless near-misses: they are effective spoofs that nothing here reliably catches.

Robustness

A simulation can produce results that only hold for the exact settings someone picked. Three follow-up experiments try to break the findings.

A different market

The core experiment was repeated in four other markets: fewer or more traders who read the book, a thinner book, and impatient traders who cancel sooner. Each claim was written down first and then checked in every market.

Two claims fail somewhere, and both failures teach something. In other markets plain PCA catches up to of the largest spoofs, so "untaught detectors catch few spoofs" does not survive. But it flags honest orders just as often, so the claim that matters, that they cannot tell the two apart, holds in every market. And with few book-reading traders the wall barely helps the spoofer (+ points, not significant): spoofing only works when someone is watching the book.

A finer view of time

Perhaps the sequence detectors failed because a spoof happens inside one or two 5-step bars. So the features were recomputed every single step, and the sequence detectors read the last 24 steps. They still could not separate spoofs from normal trading (AUC for the LSTM) or from honest orders (). The bar size was not the problem.

Orders instead of the market

Every untaught detector so far described the market: depth, counts, flows. The rule works because it describes individual orders. So a final Isolation Forest was trained on every order in normal trading, each described by four plain facts: its size, how long it lived, how much of it traded, and how far from the best price it sat.

Figure 7 What an untaught detector flags, order by order

Figure 7. Share of each kind of order that the order-level Isolation Forest flagged, with the threshold set so that 1 ordinary order in 1,000 is flagged. It was trained on normal trading only.
Show the data for Figure 7

It noticed that spoof orders are unusual compared with ordinary ones (AUC ), but it ranked every honest large order above every spoof (AUC ). Take size out of its description and spoofs stop standing out almost entirely (AUC against ordinary orders). Size is the only unusual thing about a spoof. Its quick cancellation looks exactly like the market makers who cancel all day, while an honest large order is big and patient, which is rarer.