Never make "upgrading to a new release" the reason why "old backtest results cannot be exactly reproduced"
Never make "upgrading to a new release" the reason why "old backtest results cannot be exactly reproduced"
26 Nov 2024, 16:17
Latest example was the more realistic treatment of backtest swap in v5 vs v4 but no compatibility or fixed rate option - making old saved backtest results almost useless for development because there's no way to tell whether some change in backtest results was due to a subtle cBot logic alteration, or due to the version of cTrader & its treatment of swap.
More importantly: making backtest results unconditionally different means there's no opportunity to examine & understand EXACTLY why the backtest result is different after a cBot logic change - by rerunning the old and new logic with extra debug log messages etc for comparison with stored much older results.
We already can locked down the price data by using csv files, and the spread by setting a fixed rate, so why not also the swap, with also an option to handle it the old way.
That would leave old backtest reproducability still vulnerable to changes in commision rate, so can we set that too please eventually.
(See https://ctrader.com/forum/ctrader-algo/45577/#post-115250 & many other forum items bemoaning backtest results changing after upgrade, and possible lack of confidence that that instills.)
Replies
martins
27 Nov 2024, 12:11
( Updated at: 27 Nov 2024, 12:36 )
RE: Never let "upgrading to a new release" be the reason why "old backtest results cannot be exactly reproduced"
Just need option for a fixed swap rate and maybe a ‘do it the v4 way’ option. Also clarity about how to control what account currency exchange rate is used in backtest.
Commission rate is covered, can set own rate for backtest (I remembered after posting but there's no way to edit while ‘awaiting moderation’, hint!) - it's just swap rate that needs a way for setting own fixed rate and maybe a compatibility mode for “like v4” (where I think swap was not subtracted from equity until the end.
Also, if cTrader changes the way account currency exchange rate is calculated (from ‘fixed rate at backtest end date’ to ‘same calc as live’?) then please: there would need to be a compatibility mode option for 'doing it the old way', otherwise old backtests become useless again.
Or is that what the button “download for historical data for additional symbols to accurately convert profits to account currency” is for??
In which case: Is ‘leaving that button unticked’ effectively the compatibility option? - please be clearer with documentation on anything that affects backtest results.
@martins
Salileh
27 Nov 2024, 09:44
cTrader.Automate. Host.Dispatcher. Exceptions.AutomateDispatcherUnhandledException' was thrown.
Hi,
I suffered too much trying to re-back testing my earlier developed cbots on v, 5.040, and the logs indicated the sentence under subject, but with thorough follow up with cTrader Spotwere technical team it was resolved with the recent ctrader update v. 5.0.46. Hoping it will not cause same trouble for future ctrader updates.
@Salileh