Home › Academy › Indicators & Strategy › Backtesting without fooling yourself
📊 Indicators & Strategy · Building a strategy

Backtesting without fooling yourself

4 min read · Lesson 9 of 11

The basics lesson on backtesting covered manual testing. This lesson focuses on systematic, rule-based backtesting of indicator strategies — using platform strategy testers, reading the reports correctly, and avoiding the statistical traps that make so many backtests look better than reality.

Tools for automated backtesting

ToolLanguageNotes
MetaTrader 4/5 Strategy TesterMQL4/MQL5Widely used with forex brokers; MT5 supports multi-currency and real-tick testing
TradingView Strategy TesterPine ScriptEasy to start; check settings for commission and slippage
Python (e.g. backtrader, vectorbt, pandas)PythonMost flexible; requires programming and clean data
SpreadsheetFormulasGood for simple daily-bar strategies

Data quality matters

  • Enough history: several years covering trends, ranges and crises.
  • Correct timezone: daily candles and session filters depend on it.
  • Spreads and commissions: include realistic costs — for gold, a spread of $0.20–0.40 plus commission if applicable.
  • Intrabar detail: if your stop and target can both be hit within one candle, the tester must know which came first. Use lower-timeframe or tick data where possible.

Reading a backtest report

MetricWhat to look for
Number of tradesAt least 100 for meaningful statistics
Net profit and expectancyPositive after costs; expectancy in R
Profit factorGross profit ÷ gross loss; above about 1.3 is often considered solid for robust strategies
Maximum drawdownCan you tolerate it emotionally and financially?
Longest losing streakCompare with the losing streak calculator
Results by yearConsistent, or one lucky year?

A strategy with profit factor 3.0 and 25 trades is less trustworthy than one with profit factor 1.4 and 400 trades.

The overfitting trap

With enough parameters, you can make any strategy look perfect on past data. Signs of overfitting:

  • Results change dramatically when a parameter moves slightly (for example RSI 14 profitable, RSI 13 and 15 losing).
  • Many rules, each added to remove specific losing trades.
  • Excellent in-sample results, poor out-of-sample results.

Robustness test: vary each parameter by ±20%. A robust strategy stays profitable across a range of settings — a "plateau" rather than a single "peak".

In-sample and out-of-sample

  1. Split your data: for example 2018–2023 for development (in-sample), 2024–2026 for testing (out-of-sample).
  2. Develop and adjust rules only on in-sample data.
  3. Run the final rules once on out-of-sample data — no changes allowed afterwards.

Walk-forward analysis repeats this process across rolling windows, giving a more realistic view of how the strategy would have adapted over time.

Monte Carlo simulation

A backtest shows one sequence of trades. Monte Carlo testing shuffles the order of trades thousands of times to show the range of possible drawdowns and outcomes. If 5% of simulations show a 30% drawdown, you should be prepared for that possibility even if the original backtest showed only 15%. See risk of ruin calculator for a related view.

Common biases

  • Look-ahead bias: using information not available at the time (for example, today's close to decide an entry earlier today).
  • Survivorship bias: testing stock strategies only on companies that still exist today.
  • Selection bias: testing only the periods or instruments where the idea seemed to work.

Example workflow

  1. Write rules — see strategy rules.
  2. Code them in your chosen tool with realistic costs.
  3. Run in-sample; examine trades visually to ensure the code matches your intent.
  4. Check robustness (±20% parameters).
  5. Run out-of-sample once.
  6. Run Monte Carlo for drawdown expectations.
  7. Move to forward testing.

Frequently asked questions

Do I need to code to backtest? No — manual and spreadsheet testing work for simple strategies. Coding becomes valuable when you test many variations or years of data.

Why do live results differ from backtests? Slippage, execution delays, changing markets, and human behaviour. Expect live results to be somewhat worse and size accordingly.

How many years of data should I test? For daily strategies, at least five to ten years. For intraday strategies, one to three years usually gives enough trades, as long as different market conditions are included.

Key takeaways

  • Use quality data with realistic costs and enough history.
  • Judge reports by trade count, expectancy, profit factor, drawdown and consistency.
  • Avoid overfitting: robustness tests, out-of-sample and walk-forward analysis.
  • Use Monte Carlo to understand the range of possible drawdowns.

Profit factor thresholds are general guidelines, not guarantees. Education only, not investment advice.

Educational content by CROZAX. Not financial advice. Trading involves risk.