The Probability of Backtest Overfitting (PBO), Explained
What PBO actually measures, how the gate is computed, and why 0.15 is the line.
The question PBO answers
Walk-forward efficiency and Monte Carlo both ask "did this specific strategy hold up out of sample?" PBO asks a different, sharper question: if you had tried every parameter combination you tested and picked the best one on in-sample performance, what is the probability that pick would rank below the median out-of-sample? That is a description of the selection process, not just the final strategy — and it is the question that actually explains why backtested edges disappear in live trading.
The framework comes from Bailey, Borwein, López de Prado and Zhu's 2015 paper "The Probability of Backtest Overfitting" (Journal of Computational Finance), which introduced combinatorially symmetric cross-validation (CSCV) as a practical way to estimate it.
How the gate is computed
The engine splits your trial history into subsets, forms every symmetric train/test combination, and for each one checks whether the configuration that ranked best in-sample also ranked above the out-of-sample median. PBO is the fraction of combinations where that check fails.
The scorecard's gate is PBO < 0.15. Below that line, whatever edge survived is unlikely to be a pure artefact of how many configurations you tried before landing on this one. Above it, the strategy is failing for a reason walk-forward and Monte Carlo alone won't show you: you tested enough variations that a good-looking one was inevitable.
PBO rises mechanically with the number of configurations tested against the same data — this is why the platform tracks every run, not only the ones you kept.
Why this is a different failure mode than curve-fitting one strategy
A single overfit parameter set is a bug you can spot by eye — an entry threshold tuned to the exact cent of a historical spike. A high PBO is a property of the search, not any one result: run enough RSI thresholds, moving-average pairs, or stop-loss levels against the same 18 months of BTC data, and the best-looking one will look good even if none of them have a real edge. PBO is the honest accounting for that process.
This is also why the scorecard marks a run "insufficient" rather than fabricating a PBO value when there isn't enough trial history to estimate it credibly — a confident-looking number computed from too little data would be worse than no number at all.