/ Check 8 · Window, venue, costs

Why can’t I reproduce this backtest?

Usually because the report left out a setting that decides the result. Six of them have to be stated: date range, symbol, exchange, commission, slippage and margin. Here is why each one matters, with one worked example and no code.

The short answer

A backtest is a claim that a set of rules, run on a stated stretch of a stated market at stated costs, produced a stated result. Take away any of those and nobody, including the author, can run it again and get the same numbers. A result that cannot be reproduced is not evidence; it is a screenshot. The check takes a minute: look for six settings, and treat any one that is missing as a reason to stop.

The six settings

  • Date range. A start date and an end date. “Two years of data” is not a window until you know which two years: a trend rule tested only through one long rally has not been tested on anything else.
  • Symbol. The exact instrument and timeframe. The same name can mean a spot pair, a perpetual future or an index, and they do not trade alike.
  • Exchange. Where the prices came from. Two venues' candles for the same symbol differ in their highs, lows and gaps, and a rule that trades on wicks can fire on one and not the other.
  • Commission. The fee per side, or a plain statement that it was set to zero.
  • Slippage. How far from the signal price each fill was assumed to happen. Zero slippage means every order filled at exactly the price on the chart.
  • Margin. The leverage, if any. It does not change the trades; it changes what they do to the account.

A worked example: one report, three results

A report shows 200 trades with an average of +0.15% a trade before costs. Added up, that is 200 × 0.15% = +30%. The report does not say what commission or slippage it used. Here is what the same 200 trades are worth at three cost settings:

  • 0.05% a side, no slippage. The round trip costs 0.10%, leaving 0.05% a trade: about +10% over the 200 trades.
  • 0.05% a side plus 0.02% slippage a side. The round trip costs 0.14%, leaving 0.01% a trade: about +2%.
  • 0.10% a side, no slippage. The round trip costs 0.20%, and every trade loses 0.05% on average: about −10%.

Same signals, same 200 trades, and the sign of the result depends on a number the report left out. The edge is gone at a round trip of 0.15%, or 0.075% a side. If your own fees and fills sit near that, the backtest is a bet on execution. The cost-in-R calculator does this arithmetic in R instead of percent, and the turnover-cost calculator does it for a portfolio that trades every day.

Margin works on the other side of the ledger. At 5× margin the account moves five times as far as the position, so the +10% case becomes about +50% on paper, and a 20% move against one open position is the whole account. A report that quotes the +50% without the 5× is quoting the leverage, not the rules.

How to run the check on any report

1. Find the six settings. On TradingView they sit in the strategy's Properties tab and the chart header; in a vendor's report they should be on the same page as the result. Write each one down. A setting you had to guess is a setting that is missing.

2. Treat zero as a statement. Commission 0 and slippage 0 are allowed, but then the result is before costs and should say so. Check 3, the cost that zeroes it, tells you how much room the edge has.

3. Check the window against the claim. A length on its own (“145 days”) is not a window; a length with an end date (“145 days to 6 October”) is. Ask whether the window covers more than one kind of market, and whether the result is quoted for the whole window or for a flattering part of it.

4. Ask the trade count. Six settings and no trade count still leaves the result unweighable. Check 1, how many trades, is the next stop.

Why this is the check people skip

Nothing on the equity curve tells you a setting is missing. The chart looks the same at zero cost as at real cost, and the same at 1× as at 5× until you read the scale. That is why this check is a list to tick rather than something you can see: if any of the six is absent, the result cannot be checked, and a result that cannot be checked has not earned a forward test.

This is check 8 of 10

The one-page Backtest Verification Checklist, free. Confirm the email and the PDF opens straight away; one worked check each Tuesday after that.

How these numbers were worked out

Every figure is arithmetic on one made-up report, not a result of any system: 200 trades, average +0.15% a trade before costs. Totals are simple sums of per-trade returns, without compounding. Round-trip cost is twice the per-side cost: 2 × 0.05% = 0.10%; 2 × (0.05% + 0.02%) = 0.14%; 2 × 0.10% = 0.20%. Net per trade is 0.15% minus the round trip: 0.05%, 0.01% and −0.05%, which over 200 trades is +10%, +2% and −10%. The edge is zero at a round trip equal to the 0.15% average, 0.075% a side. At 5× margin returns on the account are five times the position's, so +10% becomes +50%, and a move of 100% ÷ 5 = 20% against the position equals the account. Nothing here is investment advice.

Historical simulations are not live results. Nothing on this site is investment advice.