Why Backtesting Lies: The 3 Biases That Inflate Every Strategy's Results
Every trader who has ever backtested a strategy has felt that moment of excitement — a Sharpe ratio of 2.4, a win rate of 61%, a smooth equity curve that trends upward with almost no drawdown. Then they run it live. Within six weeks, the account is down 18%.
This is not bad luck. It is almost always one of three systematic errors that are baked into the backtest from the start.
1. Look-Ahead Bias
Look-ahead bias is the most dangerous — and the most common — error in retail backtesting. It occurs when a backtest uses future information that would not have been available at the time of the trade.
The most frequent example: a strategy fires a signal on bar N's close and executes the trade at bar N's close price. In reality, by the time bar N closes and your system processes the signal, the next bar has already started. You cannot fill at bar N's close — you fill at bar N+1's open, with some slippage on top.
This may seem like a minor issue. It is not. In volatile markets, the difference between bar N's close and bar N+1's open can be 0.2%, 0.5%, or in a news event, 2%. Across thousands of trades, this difference entirely explains many strategies that appear profitable in backtests but fail live.
The fix is mechanical: always execute at next-bar open. Quantinger's backtesting engine enforces this by design. Signals fire on bar close; fills happen at the next bar's open price. This is not configurable — it is the only correct behavior.
2. Survivorship Bias
Survivorship bias affects any backtest that uses a fixed universe of assets. If you run a strategy on "the top 20 cryptocurrencies by market cap" using today's list, you are only testing on assets that survived to become large. You are automatically excluding every coin that crashed and delisted.
In crypto, this is severe. Hundreds of tokens from the 2017-2021 era no longer exist. If your strategy included any of them, you would have had significant losses. But if you only backtest on coins that are still trading today, those losses never appear.
The solution is to use rolling universes — assets that were relevant at each point in time, not assets that are relevant today. This is a data infrastructure problem, not a strategy problem. It requires storing historical universe membership, not just historical prices.
Quantinger currently focuses on the major pairs (BTC, ETH, BNB, SOL, and others) that have continuous trading history. For these assets, survivorship bias is minimal. Avoid running broad altcoin strategies on short history windows — the survivorship effect is amplified when you only have 6 months of data.
3. Overfitting (Data Mining Bias)
Overfitting is subtler than the other two biases. A strategy is overfit when its parameters are tuned to work specifically on historical data rather than reflecting a genuine market dynamic.
Here is how it typically happens: a trader tests 200 parameter combinations of an RSI mean-reversion strategy. One combination — RSI(23) with a threshold of 27 — shows a Sharpe ratio of 1.9. The trader concludes that RSI(23) with a threshold of 27 is the optimal setup.
But why would RSI(23) be better than RSI(14) or RSI(21)? There is no theoretical reason. The parameter was found by searching until something worked on the historical sample. When the market regime changes, RSI(23) with threshold 27 will perform like any other random combination.
The antidote is out-of-sample testing. Specifically, walk-forward analysis: divide your data into three windows chronologically — train (50%), validation (25%), test (25%). Optimize on train. Measure on test. The test window performance, using data the optimization never touched, is your honest performance estimate.
If your test window Sharpe ratio is significantly below your train Sharpe ratio, the strategy is overfit. No amount of additional parameter tuning on the same dataset will fix this. You need a different strategy, or you need to test on a longer time window.
The Bottom Line
A backtest is not evidence of edge. It is a hypothesis that requires rigorous validation. Walk-forward analysis, out-of-sample testing, and conservative slippage assumptions are the minimum bar for taking a strategy seriously.
If your backtest does not explicitly handle all three of these biases, the equity curve is fiction. The good news: a well-constructed backtest with realistic assumptions that still shows positive metrics is genuinely useful. It does not guarantee live performance — nothing does — but it meaningfully increases the probability that you are trading something real.