Is my backtest overfit?
Your backtest is probably overfit if its results collapse on data you didn't tune on, if small input changes swing the outcome, or if you tried many versions and kept the best. Check it out of sample, count every version you tested, deflate the Sharpe ratio for those tries and demand enough trades for every input you tuned.
Arnau Lavernia · Founder of Elave · Last updated:
What does overfit mean?
A backtest is overfit when its rules have learned the noise of one stretch of history instead of a pattern that repeats. Each filter added after a losing trade, and each input tuned until the curve looks smooth, fits the past more closely and the future less.
Bailey, Borwein, López de Prado and Zhu (2017) show why ordinary holdout testing is not enough on its own: when you try many versions on the same history, the best-looking one tends to do worse on new data than its backtest promised.
How do I tell if a backtest is overfit?
- Split the history before you look. Tune on the first part, then judge the frozen rules on the last part, roughly 70% then 30%.
- Compare the two parts. If the later trades earn less than half of what the earlier ones did, treat the edge as overfit until shown otherwise.
- Count every version you tried, including the ones you ran in TradingView and never saved.
- Deflate the Sharpe ratio for that count. The more versions, the higher the bar a real edge must clear.
- Nudge each input a step or two either side. A real edge weakens gently; an overfit one falls off a cliff.
- Count trades per tuned input. Five tuned inputs on 60 trades leaves about a dozen trades of evidence for each choice.
- Add realistic commission and slippage. Edges that live in the last tick disappear first.
- Forward-test the frozen rules on a demo account before trusting a funded one.
How much does trying more versions inflate the best result?
Even with no edge at all, the best of many versions looks good by chance. The table shows the best Sharpe ratio luck alone is expected to produce, in standard errors of the estimate, from the formula in Bailey and López de Prado (2014). It is the same calculation Elave's report runs.
| Versions tried | Best result luck alone produces | What it means |
|---|---|---|
| 1 | 0.00 standard errors | No selection: the raw result stands |
| 2 | 0.52 standard errors | A modest result can already be luck |
| 5 | 1.19 standard errors | A modest result can already be luck |
| 10 | 1.57 standard errors | A modest result can already be luck |
| 20 | 1.90 standard errors | Only a clearly larger result stands out |
| 50 | 2.28 standard errors | Only a clearly larger result stands out |
| 100 | 2.53 standard errors | Only a clearly larger result stands out |
| 1,000 | 3.26 standard errors | Most impressive-looking results are luck |
What is the deflated Sharpe ratio?
The deflated Sharpe ratio is the probability that a strategy's true Sharpe ratio beats the best one luck alone would produce across the versions you tried. It also adjusts for the number of trades and for skewed, fat-tailed results. A value near 1 means the result clears the luck bar; near 0.5 or below, it doesn't.
What is the probability of backtest overfitting?
The probability of backtest overfitting, or PBO, estimates how often the version that ranks best in-sample ranks below the median out-of-sample. It uses combinatorially symmetric cross-validation, which needs the returns of every version you tried, not just the winner. Elave's report does not compute PBO, because a single exported list of trades doesn't contain the other versions.
In-sample, out-of-sample and walk-forward: what is the difference?
All three answer "does this hold on data it has not seen?" with a different degree of strictness. Walk-forward analysis repeats the out-of-sample test window after window, so it tests the tuning process, not only one tuned result.
| Test | What it is | What it can show | Typical trap |
|---|---|---|---|
| In-sample | Rules built and measured on the same data | That the rules can describe the past | Looks best exactly when it is most overfit |
| Out-of-sample | Rules frozen, then measured on data they never saw | Whether the result survives new data | Peeking at it, then tuning again |
| Walk-forward | Repeated: tune on a window, test on the next, roll forward | Whether the tuning process itself holds up | Few trades in each test window |
| Forward test | Same rules on a demo or small live account | Real fills, costs and your own execution | Stopping after a lucky first week |
What is a good Sharpe ratio for a trading strategy?
There is no universal threshold, because a backtest Sharpe ratio is in-sample and undeflated. What matters is whether it survives the number of versions you tried, an out-of-sample period and real costs. Elave's report treats an annualised Sharpe above 3 as a red flag and shows the ratio as "Not trusted" while that flag fires.
How does Elave's report check for overfitting?
- A chronological 70/30 split of your trades, with an overfit warning when the last 30% keeps less than half of the earlier average trade.
- A deflated Sharpe ratio that counts each version you save in Elave as one more try. Versions tried only in TradingView are invisible to it.
- No letter grade under 300 trades, and a sample meter that says so.
- Red flags such as a Sharpe above 3, a profit factor above 3 on fewer than 200 trades, or no losing month, which turn the Sharpe and Sortino ratios into "Not trusted".
- A what-if simulator whose rule changes show no headline number until they hold on the last 30% of trades.
FAQ
How can I tell if my backtest is overfit?
Compare the result on data you did not tune on with the result on data you did. A big drop is the clearest sign. Other signs: small input changes swing the outcome, you tried many versions, there are few trades per tuned input, or the equity curve looks too smooth to be real.
What is the deflated Sharpe ratio in simple terms?
It is a Sharpe ratio with the luck taken out. Bailey and López de Prado (2014) compute the probability that a strategy's true Sharpe beats the best result chance alone would give across all the versions you tried, adjusted for sample size and fat tails. More versions tried means a stricter bar.
How many trades do I need per parameter?
There is no fixed rule, but the evidence is shared between every input you tuned. Sixty trades for five tuned inputs is about a dozen trades per choice, which proves very little. Fewer inputs, more trades and a separate out-of-sample period all make the same result more believable.
Does out-of-sample testing prevent overfitting?
Only if you use it once. Each time you look at the out-of-sample result and go back to tune again, that data becomes part of the sample you fitted. Keep a final stretch of history untouched until the rules are frozen, then forward-test on a demo account.
What is walk-forward analysis?
Walk-forward analysis tunes a strategy on one window of history, tests it on the next unseen window, then rolls both windows forward and repeats. The joined test windows show how the tuning process performs on data it never saw, which is a stricter check than a single in-sample result.
Can a strategy pass every test and still fail live?
Yes. Tests on past data cannot include your own execution, future market regimes or every cost. A backtest is a filter that rejects bad ideas cheaply, not proof of a good one. Forward-test on demo or at a small size and compare the results with the backtest's range.
Keep going
Sources
- Bailey & López de Prado (2014), The Deflated Sharpe Ratio, Journal of Portfolio Management · checked 2 Nov 2026
- Bailey, Borwein, López de Prado & Zhu (2017), The Probability of Backtest Overfitting, Journal of Computational Finance · checked 2 Nov 2026
- TradingView — Pine Script: repainting · checked 1 Nov 2026
Related
About Elave
Elave (elavefx.com) is a free trading platform for retail and prop-firm traders: an auto-synced trading journal, Guardian rule checks on MetaTrader 5, an AI Coach that reads your own trades, a report for TradingView Strategy Tester backtests, a Telegram trading desk and application-only coaching (Elave Bootcamp). The platform is free with no subscription and no card; Elave is funded by partner-broker (introducing-broker) relationships.
Educational content, not investment advice. Elave makes no performance, win-rate, pass-rate or income claims. Trading leveraged products carries a high risk of loss.