Backtesting and optimization problem - different results on identical parameters

PanagiotisCharalampous
27 Apr 2020, 09:40

Hi victor.major,

Can you please provide the complete cBot code and steps to reproduce the issue?

Best Regards,

Panagiotis 

Join us on Telegram

victor.major
27 Apr 2020, 12:17

RE:

Thank you for replying Panatiotis,

this happens with at least one closed source cBot. Regarding an open source example, the "Sample Breakout cBot" exhibits this behaviour. 

Steps to reproduce are as above:

1. Optimise

2. Apply

3. Get a vastly different result compared to the optismisation run whose settings were applied.

V.

PanagiotisCharalampous said:

Hi victor.major,

Can you please provide the complete cBot code and steps to reproduce the issue?

Best Regards,

Panagiotis 

Join us on Telegram

PanagiotisCharalampous
27 Apr 2020, 12:20

Hi victor.major,

For the Sample Breakout Bot, can you please provide me with the following?

1) Broker

2) Symbol

3) Oprimization Dates.

4) Parameters set that gives different results

Best Regards,

Panagiotis 

Join us on Telegram

victor.major
27 Apr 2020, 12:38

RE:

1. IC Markets

2. US2000 (Russel index)

3. 26/03 - 26/04

4. These settings are not good, but illustrate the problem. 

[ChartParameters]
Symbol = US2000
Timeframe = m1

[cBotParameters]
Quantity = 1
StopLossInPips = 1990
TakeProfitInPips = 550
Source = Open
BandHeightPips = 305
Deviations = 1.5
Periods = 15
MAType = Simple
ConsolidationPeriods = 1

PanagiotisCharalampous said:

Hi victor.major,

For the Sample Breakout Bot, can you please provide me with the following?

1) Broker

2) Symbol

3) Oprimization Dates.

4) Parameters set that gives different results

Best Regards,

Panagiotis 

Join us on Telegram

victor.major
01 May 2020, 00:50

RE:

Any updates regarding this?

PanagiotisCharalampous said:

Hi victor.major,

For the Sample Breakout Bot, can you please provide me with the following?

1) Broker

2) Symbol

3) Oprimization Dates.

4) Parameters set that gives different results

Best Regards,

Panagiotis 

Join us on Telegram

PanagiotisCharalampous
04 May 2020, 09:19

Hi victor.major,

It is a bug and it will be fixed in an upcoming update. 

Best Regards,

Panagiotis 

Join us on Telegram

victor.major
25 May 2020, 15:23

RE:

PanagiotisCharalampous said:

Hi victor.major,

It is a bug and it will be fixed in an upcoming update. 

Best Regards,

Panagiotis 

Join us on Telegram

Hi Panagiotis,

when do you anticipate this update to come? I am still unable to obtain sane optimization/backtesting results. The latest optimization attempt gives me a USD 3.5k profit in a given period, while the exactly the same, identical settings (both using tick data) in backtesting results in a USD 5k loss. This is not really useful.

V.

PanagiotisCharalampous
25 May 2020, 15:29

Hi victor.major,

This will be fixed in v3.8. The beta will be released soon. The problem is that when you assign the values to backtesting Close values are set as Source instead of Open. Please try changing the source manually to Open and let me know if this resolves the problem.

Best Regards,

Panagiotis 

Join us on Telegram

victor.major
25 May 2020, 16:02

RE:

PanagiotisCharalampous said:

Hi victor.major,

This will be fixed in v3.8. The beta will be released soon. The problem is that when you assign the values to backtesting Close values are set as Source instead of Open. Please try changing the source manually to Open and let me know if this resolves the problem.

Best Regards,

Panagiotis 

Join us on Telegram

Ok that is great. I look forward to v3.8. The same problem occurs with tick data and not just with the example cBot, but also with the closed source ones.

V.

dave.anderson.consulting
25 Aug 2023, 13:58 ( Updated at: 26 Aug 2023, 14:46 )

This issue has never been resolved since it was reported. We are now 4 years later and still no fix on cTrader version 4.8.23.
I needed to optimize a client's cBot and ran into this bug today.
To make sure it's not my fault I grabbed the above passed cBot and tested it and saw that there is still a difference between Optimization and Backtest results. Exactly as it was reported here in 2019.

Reply to standard reply of support “Share the cBot with us so that we can reproduce”:

See the same issue with cBot code in here: https://ctrader.com/forum/cbot-support/15410

PanagiotisChar
26 Aug 2023, 14:56 ( Updated at: 26 Aug 2023, 15:00 )

RE: Backtesting and optimization problem - different results on identical parameters

dave.anderson.consulting said: 

This issue has never been resolved since it was reported. We are now 4 years later and still no fix on cTrader version 4.8.23.
I needed to optimize a client's cBot and ran into this bug today.
To make sure it's not my fault I grabbed the above passed cBot and tested it and saw that there is still a difference between Optimization and Backtest results. Exactly as it was reported here in 2019.

Reply to standard reply of support “Share the cBot with us so that we can reproduce”:

See the same issue with cBot code in here: https://ctrader.com/forum/cbot-support/15410

Hi there,

The same symptom does not mean that it has the same cause. Better share cBot code and exact steps to reproduce.

Aieden Technologies

Need help? Join us on Telegram