Backtesting and optimization problem - different results on identical parameters
Backtesting and optimization problem - different results on identical parameters
26 Apr 2020, 10:17
Hi,
I have a problem with backtesting and optimization.
Example: I optimize a cBot's settings, get the trading statistics (for example Net profit of $1230) "Apply" these settings to the cBot settings and hit "Backtest"... and I get $281 Net profit..
Backtesting settings are both m1 data in this example, but I have the same error (with different Net profit results) when I use tick data.
What is happening and how can I fix this? It makes it difficult to impossible to optimise or backtest cBots if I cannot trust the results.
V.
Replies
victor.major
27 Apr 2020, 12:17
RE:
Thank you for replying Panatiotis,
this happens with at least one closed source cBot. Regarding an open source example, the "Sample Breakout cBot" exhibits this behaviour.
Steps to reproduce are as above:
1. Optimise
2. Apply
3. Get a vastly different result compared to the optismisation run whose settings were applied.
V.
PanagiotisCharalampous said:
Hi victor.major,
Can you please provide the complete cBot code and steps to reproduce the issue?
Best Regards,
Panagiotis
@victor.major
PanagiotisCharalampous
27 Apr 2020, 12:20
Hi victor.major,
For the Sample Breakout Bot, can you please provide me with the following?
1) Broker
2) Symbol
3) Oprimization Dates.
4) Parameters set that gives different results
Best Regards,
Panagiotis
@PanagiotisCharalampous
victor.major
27 Apr 2020, 12:38
RE:
1. IC Markets
2. US2000 (Russel index)
3. 26/03 - 26/04
4. These settings are not good, but illustrate the problem.
[ChartParameters]
Symbol = US2000
Timeframe = m1
[cBotParameters]
Quantity = 1
StopLossInPips = 1990
TakeProfitInPips = 550
Source = Open
BandHeightPips = 305
Deviations = 1.5
Periods = 15
MAType = Simple
ConsolidationPeriods = 1
PanagiotisCharalampous said:
Hi victor.major,
For the Sample Breakout Bot, can you please provide me with the following?
1) Broker
2) Symbol
3) Oprimization Dates.
4) Parameters set that gives different results
Best Regards,
Panagiotis
@victor.major
victor.major
01 May 2020, 00:50
RE:
Any updates regarding this?
PanagiotisCharalampous said:
Hi victor.major,
For the Sample Breakout Bot, can you please provide me with the following?
1) Broker
2) Symbol
3) Oprimization Dates.
4) Parameters set that gives different results
Best Regards,
Panagiotis
@victor.major
PanagiotisCharalampous
04 May 2020, 09:19
Hi victor.major,
It is a bug and it will be fixed in an upcoming update.
Best Regards,
Panagiotis
@PanagiotisCharalampous
victor.major
25 May 2020, 15:23
RE:
PanagiotisCharalampous said:
Hi victor.major,
It is a bug and it will be fixed in an upcoming update.
Best Regards,
Panagiotis
Hi Panagiotis,
when do you anticipate this update to come? I am still unable to obtain sane optimization/backtesting results. The latest optimization attempt gives me a USD 3.5k profit in a given period, while the exactly the same, identical settings (both using tick data) in backtesting results in a USD 5k loss. This is not really useful.
V.
@victor.major
PanagiotisCharalampous
25 May 2020, 15:29
Hi victor.major,
This will be fixed in v3.8. The beta will be released soon. The problem is that when you assign the values to backtesting Close values are set as Source instead of Open. Please try changing the source manually to Open and let me know if this resolves the problem.
Best Regards,
Panagiotis
@PanagiotisCharalampous
victor.major
25 May 2020, 16:02
RE:
PanagiotisCharalampous said:
Hi victor.major,
This will be fixed in v3.8. The beta will be released soon. The problem is that when you assign the values to backtesting Close values are set as Source instead of Open. Please try changing the source manually to Open and let me know if this resolves the problem.
Best Regards,
Panagiotis
Ok that is great. I look forward to v3.8. The same problem occurs with tick data and not just with the example cBot, but also with the closed source ones.
V.
@victor.major
dave.anderson.consulting
25 Aug 2023, 13:58
( Updated at: 26 Aug 2023, 14:46 )
This issue has never been resolved since it was reported. We are now 4 years later and still no fix on cTrader version 4.8.23.
I needed to optimize a client's cBot and ran into this bug today.
To make sure it's not my fault I grabbed the above passed cBot and tested it and saw that there is still a difference between Optimization and Backtest results. Exactly as it was reported here in 2019.
Reply to standard reply of support “Share the cBot with us so that we can reproduce”:
See the same issue with cBot code in here: https://ctrader.com/forum/cbot-support/15410
@dave.anderson.consulting
PanagiotisChar
26 Aug 2023, 14:56
( Updated at: 26 Aug 2023, 15:00 )
RE: Backtesting and optimization problem - different results on identical parameters
dave.anderson.consulting said:
This issue has never been resolved since it was reported. We are now 4 years later and still no fix on cTrader version 4.8.23.
I needed to optimize a client's cBot and ran into this bug today.
To make sure it's not my fault I grabbed the above passed cBot and tested it and saw that there is still a difference between Optimization and Backtest results. Exactly as it was reported here in 2019.Reply to standard reply of support “Share the cBot with us so that we can reproduce”:
See the same issue with cBot code in here: https://ctrader.com/forum/cbot-support/15410
Hi there,
The same symptom does not mean that it has the same cause. Better share cBot code and exact steps to reproduce.
Need help? Join us on Telegram
@PanagiotisChar
PanagiotisCharalampous
27 Apr 2020, 09:40
Hi victor.major,
Can you please provide the complete cBot code and steps to reproduce the issue?
Best Regards,
Panagiotis
Join us on Telegram
@PanagiotisCharalampous