Successful backtests don't guarantee profit.
Successful backtests don't guarantee profit.
05 Apr 2019, 23:08
The chart below looks quite stable huh? the sample size looks like its good enough to run? wrong.
This is a sim that ran from 2011 to 2014 then it stops as loss was greater than $3000.
And after that point it starts losing most of its trades and reaching 0 in 2016-2017.
So It got me thinking maybe 2011 to 2014 was easier for algos to trade? or that my strategy is missing something or maybe it's not complex enough.
I had plenty of other strategies reaching similar moment in time then it completely turns around.
and it seems like GBPUSD starts on its big free fall in 2014 at the same time. however the script has equal buy/sell order ratio with equal distances to take profit, so market overall direction changing surely shouldn't make a difference
Just posting this here make you guys aware that even 1000-3000 sample size may not be enough. test your algos in multiple symbols to confirm it works. Generally I don't think there should be a difference in settings between symbols. if it works in 1 symbol but dosen't work in 4 other symbols propably means this sim out of 10000+ you ran propably got lucky gambling at these paricular market conditions.
It may even be that this bot works in certain market volatility only. so its just a matter of finding that one thing that makes a difference. I will spend more time to find the cause of course.
Let me know what you guys think.
Replies
TradeMingZhi
08 Apr 2019, 22:32
RE: RE:
FireMyst said:
wisegprs said:
Just posting this here make you guys aware that even 1000-3000 sample size may not be enough. test your algos in multiple symbols to confirm it works. Generally I don't think there should be a difference in settings between symbols. if it works in 1 symbol but dosen't work in 4 other symbols propably means this sim out of 10000+ you ran propably got lucky gambling at these paricular market conditions.
...
Let me know what you guys think.
My algos have generally been successful, but they need 'tweaks' between symbols because every symbol has its own 'personality' if you will.
For instance, in one algo I set a trailing stop loss using the ATR. For one Forex pair that tends to fluctuate a lot, I have this set to a higher number (21); for another Forex pair that doesn't move much, I have to keep it at a lower number (around 8). If I were to set the latter pair to a higher number (like 21 instead of 8), backtesting shows I incur HUGE losses; keeping it low keeps me in profit over the longer run.
However, the algos aren't fool-proof.
Every month I run through back-testing to see that they're stil performing relatively the same, or if the symbol's 'personality' has changed, adjusting the settings accordingly.
Yeah that makes sense, I reckon if you really wanted to nerd it out a bit maybe you could find relation between historical volatility and your stop loss/ profit distances :)
Would you mind sharing your backtest profit graph from 2011? that would motivate me more perhaps!
Thanks
@TradeMingZhi
firemyst
09 Apr 2019, 03:03
( Updated at: 21 Dec 2023, 09:21 )
RE: RE: RE:
wisegprs said:
Yeah that makes sense, I reckon if you really wanted to nerd it out a bit maybe you could find relation between historical volatility and your stop loss/ profit distances :)
Would you mind sharing your backtest profit graph from 2011? that would motivate me more perhaps!
Thanks
Here are two graphs. Dates are in the capture.
I never backtest more than a year though. I only care about now because as I said, all symbols have their own 'personality' which changes through the years (like right now the GBP will probably change because of Brexit).
The graphs below are of EURUSD. All I did in my strategy was change the value (I have set up as a parameter) from a 6 to a 3 between the two runs. Today and thus far this year, using a value of 6 works perfectly; but as you can see, 8 years ago using a value of 3 for my strategy worked better as the 6 returned disasterous results.
Same code, same overall logic, just tweaked parameters, with up to a year's backtesting performed monthly to make sure the parameters are still good for the current persona of the symbol. :-)
What I actually found most interesting about this exercise though, is even though I'm using the same demo account for this back test with the same time frame, the candles are different with their open/close.
This is with IC Markets. Not sure why that would be yet as I said, same code, same symbol, same time frame, just different parameter which shouldn't affect candle opens/close.
@firemyst
firemyst
08 Apr 2019, 10:18
RE:
wisegprs said:
My algos have generally been successful, but they need 'tweaks' between symbols because every symbol has its own 'personality' if you will.
For instance, in one algo I set a trailing stop loss using the ATR. For one Forex pair that tends to fluctuate a lot, I have this set to a higher number (21); for another Forex pair that doesn't move much, I have to keep it at a lower number (around 8). If I were to set the latter pair to a higher number (like 21 instead of 8), backtesting shows I incur HUGE losses; keeping it low keeps me in profit over the longer run.
However, the algos aren't fool-proof.
Every month I run through back-testing to see that they're stil performing relatively the same, or if the symbol's 'personality' has changed, adjusting the settings accordingly.
@firemyst