What is lookahead bias?
Using information the market had not published yet — usually filling at the same close that generated the signal. One index shift can invent an edge that never existed.
Guide · Research
How backtests lie, and what an honest one looks like
Almost every strategy works on the data it was built on. The interesting question is whether it works on data it has never seen — and most of the ways people answer that question are quietly broken.
What is lookahead bias?
Lookahead bias is using information at bar t that was not available until after bar t. It is almost never deliberate, and it is almost always fatal to the result.
The classic version is off-by-one indexing: computing a signal from the daily close and then filling the order at that same close. In reality you cannot know the close until the session is over, so the fill must land on the next bar's open. One index shift can turn a losing system into a spectacular one.
- Signal computed on the close, filled at the same close
- Indicators recalculated over the full dataset and then sliced, so early bars carry information from later ones
- Restated fundamentals or adjusted prices applied to dates before the restatement existed
- A stop that gets checked against the same bar's low after using that bar's high to decide the entry
- Splitting train and test data after normalising across the whole set
What is survivorship bias?
Survivorship bias is testing on a universe that only contains what is still listed today. Delisted, acquired, and bankrupt tickers vanish from most convenient datasets, so the test never has to hold the ones that went to zero. Any long-biased strategy looks better against a universe that quietly deleted its worst outcomes.
In-sample, out-of-sample, walk-forward
Three levels of honesty:
| Method | What it does | How much to trust it |
|---|---|---|
| In-sample | Optimise and evaluate on the same data | None. This is curve fitting with extra steps. |
| Single out-of-sample split | Optimise on the first chunk, test once on the last | Some — but you only get one honest test, and looking twice spends it. |
| Walk-forward | Roll the optimise/test window forward repeatedly, always testing on unseen data | The most useful of the three, and still not a guarantee. |
Walk-forward works like this: fit parameters on a training window, test on the block immediately after it, then slide both windows forward and repeat. The reported result is the concatenation of every out-of-sample block. Because every trade in the result came from parameters chosen without seeing that trade, the equity curve is at least structurally honest.
The costs people leave out
- Slippage — assume worse fills than mid, especially on gaps and small caps.
- Commission and financing — small per trade, decisive at high frequency.
- Partial fills — a backtest fills 100% of a limit order that reality would have missed.
- Borrow availability for shorts, which the data will not tell you about.
- Your own behaviour — the backtest takes every signal; you will not.
How to read a result
Ask three questions of any backtest, including your own:
- How many trades? Under 100 out-of-sample trades, expectancy is mostly noise.
- How many parameters were tuned? Every knob is a chance to fit history. A two-parameter system with 300 trades beats a nine-parameter system with 40.
- Where are the negative results? A write-up with no failed variants is a sales page. Real research kills more ideas than it keeps.
When I ran this on my own setups, the outcome was mostly negative: only the anchored-VWAP reclaim survived — at +0.23R over 101 trades, a number my own later adversarial re-test could not reproduce; on a 129-symbol, 10-year universe the defensible figure is +0.117R over 4,933 trades (95% CI +0.057 to +0.174). A VCP variant came out breakeven, and a gap/opening-range-breakout proxy came out at −0.28R and was cut. Two filters I was confident about — regime and normalised momentum lag — both reduced expectancy and were rejected.
That is the point of the exercise: an honest test is one that is allowed to say no. Full write-up and engine →
Educational content, not financial advice. No live profit-and-loss figures are claimed anywhere on this site; backtest and walk-forward results are always labelled as such. Full terms: /terms.html
Tools referenced in this guide
- Swing Screener — the no-lookahead backtest engine and walk-forward results.
- Trading bot — the execution side, with a hard gate requiring 60 profitable paper trades before any live broker.
- R-multiple & expectancy guide — how to measure what a test produced.
FAQ
Quick answers
What is walk-forward testing?
Fit on a training window, test on the next block, slide forward, repeat. Every reported trade came from parameters chosen before that trade existed.
Why do in-sample results look great?
Because the parameters were fitted to that data. It is curve fitting with extra steps — useful for debugging, useless as evidence.
What is survivorship bias?
Testing on a universe of survivors only. Delisted and bankrupt names are missing, so long strategies look better than they were.
How many trades are enough?
100+ out-of-sample trades before expectancy means much. Under 30 you are measuring luck with a decimal point.
What gets left out of backtests?
Slippage, commissions, financing, partial fills, borrow availability — and your own willingness to actually take every signal.