The MT5 strategy tester is powerful and dangerous. Powerful because it can run thousands of parameter combinations in minutes. Dangerous because 90% of "optimized" EAs are curve-fit garbage that lose money the moment they go live.
Here's the workflow that separates real edge from data-mining illusion.
The fundamental problem
If you optimize an EA on 5 years of historical EUR/USD data, you'll find parameters that produced a great equity curve on that exact data. That doesn't mean those parameters will work on future EUR/USD. It often means the opposite.
Step 1: Reserve 30% of your data
Split your historical data into training (70%) and holdout (30%). Optimize only on the training set. Then test the optimized parameters on the holdout data the EA has never seen. If performance collapses on holdout, your parameters are curve-fit.
Step 2: Walk-forward optimization
MT5's strategy tester has a built-in walk-forward feature. Use it. It slides the optimization window forward through time, re-optimizing on each window and testing on the next out-of-sample window. If the EA only works in certain windows, it has no real edge.
Step 3: Look at the right metrics
Profit factor and net profit are vanity metrics. Look at:
- Sharpe ratio (risk-adjusted return)
- Sortino ratio (downside risk-adjusted)
- Max drawdown (the number that kills accounts)
- Recovery factor (net profit / max drawdown)
- Number of trades (less than 200 trades = statistically meaningless)
Step 4: Forward test live
After backtest + walk-forward look good, run the EA on a demo account in real time for at least 2 weeks. Real spreads, real slippage, real broker quirks. If forward test results match backtest, you have a real edge.
This is where a VPS pays off. You don't want your demo forward test to die because your laptop slept.
Step 5: Live with conservative sizing
Backtest profit factor of 1.8 will be 1.3 live. Plan for it. Start with 50% of the position size your backtest suggests. Scale up only after 100+ live trades confirm the edge.
The optimizer trap
MT5's "Slow complete algorithm" runs every parameter combination. With 5 parameters of 10 values each, that's 100,000 combinations. Statistically, at least 100 of those will look amazing by chance alone. The optimizer will pick one. You'll lose money.
Use the "Genetic algorithm" mode instead. It explores fewer combinations but tends to find robust regions, not single-point peaks.
The reality check
If your EA backtests 100% returns/year with 5% max drawdown, it's curve-fit. Real systematic forex EAs do 15-40% per year with 10-20% drawdowns. Anything claiming better is either using a flawed backtest, a leverage trick, or is straight-up lying.
Where Skyline helps
- Run multiple MT5 instances simultaneously for parallel parameter testing (Trader plan)
- Snapshot your EA configuration before changing it · easy rollback (all plans)
- Forward-test on a real VPS without burning your laptop battery
EA development is 10% coding, 90% testing methodology. Most retail EAs fail because the methodology was lazy, not because the strategy was bad.