How to evaluate a trading bot backtest: 12 questions that matter

Start by auditing the data

What instruments, timestamps, bid/ask spreads and market hours were used? Was the strategy tested with tick-level data or compressed candles? Did it assume entries at prices that were not actually available? MetaTrader’s strategy tester documentation describes different testing and optimization modes because simulation detail matters.

Separate development from verification

If parameters were optimized on the same sample used to present the final result, the output can be biased. A forward period, a genuinely held-out test sample and several market regimes are more informative than one perfectly fitted equity curve. Changing the model repeatedly after seeing the holdout also undermines independence.

Twelve-point checklist

Before comparing two bots, record the following in a small table and mark missing inputs as unknown.

• Instrument and broker contract specification.
• Data provider and sample dates.
• Spread, commission and overnight fee assumptions.
• Slippage, order rejection and market-gap model.
• Look-ahead bias and survivorship-bias controls.
• Training, validation and held-out test separation.
• Number of parameter trials and optimization objective.
• Trade count and largest position concentration.
• Equity drawdown including open positions.
• Leverage, margin and stop-out assumptions.
• Changes to model after testing.
• Independent live or paper-trading observation.

A simple decision rule

A backtest with modest returns but transparent costs and held-out evidence can be more informative than a spectacular curve with missing assumptions. We will never translate an unverified simulated return into a claim about expected profits.