How to read an honest backtest — and what to ignore

A backtest is not evidence. A backtest is a story — the story a strategy tells about itself when it is allowed to grade its own homework. Some stories are honest. Most are not. Learning to read the difference is the most useful single skill in evaluating any automated strategy, including ours.

This article walks through the five questions to ask of any backtest before you give it credibility, and the red flags that should make you walk away.

Question 1 — over what period?

The single most abused number in retail trading. A strategy that "made 47% in 2023" tells you almost nothing. A strategy that posted a positive Sharpe ratio across 2018, 2019, 2020, 2021, 2022, 2023, 2024 and 2025 — through QE, COVID, rate hikes, ZIRP exit and the AI rally — is a different conversation.

The minimum acceptable backtest period is five years for any strategy that claims to be regime-agnostic, and at least one full regime cycle for strategies that target a specific environment (trend-following, mean-reversion, range, volatility). One year of data on a trend strategy in a trending year proves nothing about how the strategy behaves when the trend ends.

Question 2 — what spread and commission?

The MetaTrader 5 Strategy Tester lets you pick the spread model — current, fixed, or random. Current uses your broker's live spread at the moment you press start. If you run a backtest at 3am on a quiet Sunday, the spread is at its daytime tight, and the report uses that number for every fill across five years. That is a fantasy.

A credible report uses realistic spread for the strategy's session — typically the broker's average for the relevant hours — and adds commission per round-turn for ECN-style brokers. Both numbers should be stated in the methodology panel.

A scalping strategy that nets 0.6 pips per trade and tests at 0.4 pip spread is one rounding error away from being a losing strategy. A swing strategy that nets 60 pips per trade barely notices a 1-pip spread move. Ask which one you are reading.

Question 3 — what about slippage?

Slippage is the gap between the price you wanted and the price you got. The MT5 tester models slippage poorly — by default, it does not model it at all unless you check the "random delay" box. A backtest with zero slippage on a strategy that places market orders during news is structurally optimistic.

A credible backtest either (a) adds a slippage budget — usually 0.5 to 2 pips per market order depending on session and instrument — or (b) uses only pending orders that fill at the requested level, with explicit handling for gaps.

Question 4 — which broker, which symbol?

EURUSD on Broker A is not EURUSD on Broker B. The spread differs, the swap differs, the trading hours differ slightly, the digit precision can differ. The biggest gap is synthetic indices — most brokers offer their own "US30" / "USTECH" symbols with proprietary tick generation, and a strategy that prints money on one broker's feed can lose money on another.

A credible report names the broker and the symbol. "USTECH on IC Markets" is a complete spec. "US30" is not.

Question 5 — what modelling quality?

MT5 reports a modelling quality percentage at the end of every backtest. Anything under 90% means the tester filled in gaps in the tick data — which means the fills you see did not happen in the order they did in real life. For any short-timeframe strategy (M1, M5, scalping), modelling quality below 99% (and ideally 100%, with real ticks) is disqualifying.

The numbers that lie

Some MT5 KPIs are more honest than others. A practical hierarchy:

The one curve that matters

Skip the KPIs for a second. Look at the equity curve. An honest equity curve looks like a noisy upward drift with visible drawdowns — periods where it goes sideways or down for weeks, then recovers. A suspicious equity curve looks like a smooth diagonal line with no drawdowns deeper than 2%. That curve was either curve-fit to the data or massaged after the fact.

Real strategies have ugly periods. The shape of a real equity curve is the shape of a strategy living through regimes it does not love. If the curve is too clean, ask which regimes have been removed from the sample.

In-sample, out-of-sample, walk-forward

A serious backtest splits the data into at least two periods: in-sample (where the strategy is tuned) and out-of-sample (held back, never seen during tuning). The OOS result is the one that matters. An IS Sharpe of 2.1 with an OOS Sharpe of 0.4 is a curve fit, not a strategy.

Better still is walk-forward optimisation: re-tune the strategy on a rolling window and evaluate on the next window forward. Repeat across the whole history. The OOS-of-each-window stitched together is the closest a backtest gets to honest. AlphaLab-AI uses walk-forward on every EA we publish.

The methodology panel — make it mandatory

Every backtest report we publish ships with a methodology panel that names: data source, modelling quality, spread model, commission, slippage budget, swap handling, in-sample vs out-of-sample split, optimisation method (if any), and the inputs used. If a report does not state these things, it is not a report — it is a screenshot.

The forward test — the only honest test

A backtest tells you what would have happened. A forward test tells you what is happening. For any strategy worth running, the live result over six months matters more than the backtest over six years. Demo-account forward tests are the cheap version; small-size live forward tests are the expensive version. Both are non-negotiable before scaling capital.

The bottom line

A credible backtest is long, traded across regimes, named to a specific broker and symbol, run on real-tick data with realistic spread and slippage, split into in-sample and out-of-sample, and shipped with a complete methodology panel. Anything less is a marketing screenshot — and you should treat it that way.

Browse our own reports at /backtests. Every one of them ships with the methodology panel above. If anything in there looks wrong to you, write to us — we publish corrections.