How to read a backtest honestly
A backtest is a description of the past under a set of assumptions — not a preview of the future. Read that way, it is one of the most useful tools a trader has. Read the other way, as a promise, it is one of the most dangerous, because a well-presented backtest is engineered to inspire exactly the confidence it cannot justify. This is a framework for reading one honestly: what to look for, what to distrust, and how to avoid fooling yourself.
The central discomfort is this. A backtest can only ever tell you how a strategy would have behaved along the single path history happened to take. The future will take a different path. The value of the exercise is not the outcome it produces but the questions it lets you ask about robustness — and those questions live almost entirely in the parts of the report nobody frames on a wall.
Read the report backwards
The eye is drawn to the headline first: the total return, the smooth rising equity curve. Train yourself to read in the opposite order. The headline is the least informative number in the document, because it is the one most easily inflated by assumptions and most easily achieved by fitting. Start instead with the things the strategy would rather you did not notice.
- Maximum drawdown and its duration — not just how deep, but how long it stayed underwater, because that is the number you would actually have to live through.
- The assumptions on costs: were commissions, spread, and slippage modelled, and modelled pessimistically, or quietly set to zero?
- Trade count: a handful of trades can produce a beautiful curve by luck, and no statistic computed on them means much.
- The shape of returns: was the result carried by a few extraordinary trades, without which the edge disappears?
- Out-of-sample behaviour: how did the strategy perform on data it was never allowed to see while being built?
The first question, always
Ask what data the strategy was tuned on, and what data it was tested on. If those are the same data, you are not looking at a test. You are looking at a memory of the past dressed as a prediction of the future.
The assumptions do the lying
Most misleading backtests are not fraudulent; they are optimistic in a hundred small, defensible-sounding ways that compound. Fills are assumed at prices you could not have received. Costs are understated or omitted. The strategy trades at the closing price using information that, in reality, was only available after the close. Each assumption seems minor. Together they can manufacture an edge that never existed outside the spreadsheet.
Two failures deserve to be named because they are so common and so quiet. Look-ahead bias is the use of information that would not have been available at the moment of the trade — a subtle leak of the future into the past. Survivorship bias is the testing of a strategy only on the instruments that still exist today, silently excluding the ones that failed, which flatters every result by removing the outcomes you most needed to see.
“A backtest does not show you what a strategy did. It shows you what a strategy did, given everything the person who built it assumed. Read the assumptions first.”
Overfitting: the flattering machine
The deepest trap is overfitting, and it is deep because it feels like diligence. A strategy with enough adjustable parameters can be tuned until it navigates the historical data almost perfectly — buying every dip that happened, avoiding every decline that occurred. The result looks like insight. It is closer to memorisation. The strategy has not learned how markets behave; it has learned what this particular slice of the past did, and that knowledge does not travel.
The tell is fragility. An overfitted strategy tends to fall apart the moment its parameters are nudged, or the moment it meets data from a period it was not built on. This is why the out-of-sample period is the single most honest part of any backtest: it is the only stretch where the strategy was flying blind, and therefore the only stretch that resembles the conditions it will actually face going forward. A result that is excellent in-sample and mediocre out-of-sample is not a good strategy with a rough patch. It is a fitted curve meeting reality.
A quiet test of honesty
Before trusting a result, ask how many variations were tried before this one was chosen. If a thousand strategies were tested and the best was presented, its stellar history may be nothing more than the luckiest draw from a large pool — impressive by selection, not by edge.
Metrics in their proper place
Summary statistics are useful precisely as long as you remember what they cannot do. A win rate tells you how often the strategy was right, but says nothing about whether the wins were larger than the losses. A risk-adjusted measure such as a Sharpe ratio describes how smooth the historical ride was relative to its returns, but a smooth past ride is no guarantee of a smooth future one, and the figure is easily flattered by the same optimistic assumptions as everything else. Every metric is a compression of a messy reality, and compression discards exactly the tail events you most need to respect.
So hold the numbers loosely. Treat a strong backtest not as a verdict but as a hypothesis that has survived one round of scrutiny and now deserves another — forward testing on unseen data, a modest live allocation, continued observation. The honest reader of a backtest is not looking to be convinced. They are looking for the reasons the result might be an illusion, and only trusting what remains after those reasons have been fairly tried.
None of this predicts whether a strategy will make money, and it is not intended to. What honest reading buys you is narrower and more valuable: protection against your own eagerness to believe a beautiful curve. The past, described carefully and under stated assumptions, is genuine evidence. Mistaken for a promise, it is the most expensive story in the market.