How to backtest a trading strategy without fooling yourself

Step 1 — Write the rules before you look

A backtest tests a rule, not a feeling. Before you open a chart, write down: what has to be true for a setup to exist (context), what triggers the entry, where the initial stop goes and why, what would prove the idea wrong, and how you exit. Then write the sentence that matters most: "I would be wrong if…". An idea that cannot fail cannot pass either.

This is what a playbook is for — one entry per idea with its rules and its lifecycle from idea to testing to live or retired. Writing the hypothesis down before the data can answer is called pre-registration in research, and it is the single habit that separates testing from storytelling. Sage's Playbook asks for it explicitly, and links every backtest session to the play it is testing.

Step 2 — Replay, don't scroll

Scrolling a historical chart is not backtesting. When the right side of the chart is visible, your eye reads the future and your "setup" quietly becomes whatever preceded a good move. The only fair way to test a discretionary strategy is to replay the market: the chart stops at a moment in time, you decide, and only then does the next candle print.

Two details make or break a replay. First, the chart must not leak future data on any timeframe — switching from 5-minute to 4-hour mid-session must still show only what you would have known, including the candle still forming. Second, fills must resolve inside the bar: a stop is hit the moment price touches it, not when a slow candle finally closes. Sage's replay workspace is built as a pure function of the replay cursor, so there is no path that returns a future candle, and stops and targets resolve on 1-minute data.

Step 3 — Split the history

Any process of adjusting rules against past data fits noise as well as signal. The defence is simple and non-negotiable: divide your history. Tune the idea on the first part (the in-sample period). When you have a version you believe in, run it on the part you never looked at (out-of-sample) — and look at that result exactly once. If it holds up, you have something. If it collapses, the in-sample result was mostly fitting.

In-sample:      2023-01 → 2025-06   (tune here, as many passes as you like)
Out-of-sample:  2025-07 → today      (look once, with the rules frozen)
Never move the boundary after seeing the out-of-sample result.

Step 4 — Measure in R, record the stop

Record every backtest trade the way you would a live one: entry, initial stop, planned target, exit, session and the play it belongs to. Express results in R (multiples of the initial risk) so trades of different sizes are comparable — see risk/reward and R-multiples. The stop at entry is the field you will regret skipping: without it you cannot compute R at all, and you cannot later ask the most valuable question in backtesting — would a different fixed target have paid better?

That question deserves an exact answer rather than an impression. For each trade, walk the real minute-by-minute path from your entry and race a fixed target (1R, 2R, 3R…) against the stop you had at fill, not truncated at where you actually exited. Sage's R:R simulator does precisely this on every backtest and live trade, and reports each target's win rate, average R, profit factor and drawdown side by side (how it works).

Step 5 — Know when the sample is big enough

A backtest with twelve trades proves nothing in either direction. Win rates from small samples swing wildly: 7 wins in 12 looks like 58%, but the plausible range is roughly 30% to 80%. The honest way to read a backtest is with a confidence interval around every rate, and to keep testing until the interval is narrow enough to act on. Thirty trades is where a pattern starts to separate from noise; a hundred is where you can begin to trust a difference between two setups.

The ranges above are Wilson intervals — the same ones Sage prints next to every rate it shows, because a number without its uncertainty invites a decision the data cannot support. More on reading these in trading statistics that matter.

Step 6 — Grade the process, not the outcome

In a backtest you will take trades that follow every rule and lose, and trades that break a rule and win. Grading by outcome teaches you the wrong lesson from both. Grade each trade on process: was the setup valid under your written rules, was the stop defined and placed where the rule says, did you exit by plan? Then separate good losses (right process, wrong outcome) from bad wins. Over a sample, process grades predict future results far better than outcomes do.

Step 7 — Decide what to do with the result

• Edge on both in-sample and out-of-sample, with a sample you trust → move the play to live at small size, and keep journaling it the same way.
• Edge in-sample, gone out-of-sample → the rules fitted noise. Simplify them, or retire the idea. Retiring a play is discipline, not failure.
• No edge anywhere, but a clear pattern in when it works (a session, a day, a condition) → that is a new, narrower hypothesis. Write it down and test it as its own play.

One more honesty check: a backtested edge is evidence, not a promise. Costs, slippage beyond spread, and your own behaviour under real risk are not in the test. Size the first live trades as if the edge might be half what the backtest suggests, and let the live journal settle the argument.