workaround Backtest LoadmoreHistory()

Created at 09 Jan 2021, 10:38
How’s your experience with the cTrader Platform?
Your feedback is crucial to cTrader's development. Please take a few seconds to share your opinion and help us improve your trading experience. Thanks!
GE

genappsforex

Joined 25.11.2019

workaround Backtest LoadmoreHistory()
09 Jan 2021, 10:38


Porting some of our proprietary indicators to cTrader is troublesome.

We need to be able to go back xx days to establish a 'tick profile'
We need history tickdata in Backtest but LoadmoreHistory() (ticks & Bars) does not work in Backtest.
As you can work on ticks in Backtest (symbol.Ask&Bid are there, but for indicators only for Bar level.) it should not be too difficult to make it publically available to cBots & indicators

Request:

 = Please make the bid&Ask ticks public available for indicators & cBots in backtest. or
 = suggest an ETA or
 = viable workaround.(not being dump & readback in different processes) or
 = open up a method like
    bool BackTest(
                         DateTime start,
                         DateTime end,
                         TimeFrame timeFrame,
                         double startBalance,
                         double commision,
                         BacktestMethod onStartMethod,
                         BacktestMethod onTickMethod,
                         BacktestMethod onBarMethod,
                         BacktestMethod onStopMethod)
;

 

Hope someone comes up with a great idea.


@genappsforex
Replies

prosteel1
09 Jan 2021, 10:58

I posted this 

 

Basically start backtesting from an earlier date and prevent trading until a later date.


@prosteel1

genappsforex
09 Jan 2021, 18:11

RE:

Thanks for caring but it does not work.

if (RunningMode == RunningMode.RealTime)
skips everything in  OnStart in backtest

 


@genappsforex

prosteel1
09 Jan 2021, 19:55 ( Updated at: 09 Jan 2021, 20:36 )

RE: RE:

genappsforex said:

Thanks for caring but it does not work.

if (RunningMode == RunningMode.RealTime)
skips everything in  OnStart in backtest

 

Yea I think I had another update to post but since I got Perma banned from the Telegram chat for posting a link to this workaround I didn't bother to update it. I haven't been able to ask questions in Telegram since. I guess posting workarounds is a banable offense lol.

I ended up adding another condition Ontick to get around the issue you may be experiencing.... My OnTick() looks like this: By adding the FirstRun check I can run the code I would usually run OnStart() only once. Just copy the OnStart() code into if (FirstRun == 0)

Using this concept we don't want to run our analysis OnStart(), we want to delay the analysis until we have sufficient bars - this is why we are delaying the normal OnStart() code until the Server.Time.Date >= AnalysisStartDate and only once if FirstRun == 0

protected override void OnTick()
        {
            if (RunningMode != RunningMode.RealTime)
            {
                if (Server.Time.Date >= AnalysisStartDate)
                {
                    if (FirstRun == 0)
                    {

 

I hope this helps let me know if it doesn't, as I have a working example, but my posted code might be a bit old and needs to be updated :)


@prosteel1

genappsforex
09 Jan 2021, 21:02 ( Updated at: 09 Jan 2021, 21:08 )

RE: RE: RE:

Thanks for the answer.
If you ask to release your ban they probably will they're not bad guys, just sometimes a bit too protective ;-)

Sorry to say but i can not make anything from your post.
I have a gmail adress (genappsforex) maybe post it here or mail it to me?


 

bool Booted=false;
DateTime ToDate;
protected override void OnTick()
{
  if (!Booted)
  {
    Booted = true;
    int _loaded = Bars.Count;
    while (ToDate < Bars[0].OpenTime && _loaded > 0)
        _loaded = Bars.LoadMoreHistory();  // Will not work in backtest :-(
  }
}

 


@genappsforex

prosteel1
09 Jan 2021, 21:23 ( Updated at: 09 Jan 2021, 22:25 )

RE: RE: RE: RE:

genappsforex said:

Thanks for the answer.
If you ask to release your ban they probably will they're not bad guys, just sometimes a bit too protective ;-)

Sorry to say but i can not make anything from your post.
I have a gmail adress (genappsforex) maybe post it here or mail it to me?


 

bool Booted=false;
DateTime ToDate;
protected override void OnTick()
{
  if (!Booted)
  {
    Booted = true;
    int _loaded = Bars.Count;
    while (ToDate < Bars[0].OpenTime && _loaded > 0)
        _loaded = Bars.LoadMoreHistory();  // Will not work in backtest :-(
  }
}

 

Your using your code which I don't want to spend any time trying to figure out. You are asking the question so I think you should spend a bit of time using my code. My code works. I know it works because I have coded it and tested it. Try my code :) It might be a bit different to your setup so may require some changes.

I know this is a very strange concept but this is a workaround, it's not pretty, it doesn't work great, it is ugly, but it works :)

Loadmorehistory wont work in Backtesting mode, the way you can get history is by starting at an earlier date and letting it run for a while before starting the analysis - this is why I have the         DateTime AnalysisStartDate; variable :)

For instance, when I do this in my bot, I start it in backtesting mode in November 2019 using ontick. And I have my AnalysisStartDate set to 19 march 2020. So it races through the period from November 2019 to march 2020 very quickly, then when it gets to March 19th 2020 it begins the analysis and takes a while to do that, but is has access to all data from November 2019 because that is the date I started it on - I place a condition to only do the analysis after 19th March 2020 is hit and so it has access to 3 months of data!

It is a strange concept I know - it's a workaround though, not supposed to be pretty

I did find that the number of bars on the lower timeframes like Tick10, Minute etc was too large, so I limit the analysis to only the last 2000 bars by using 

if (series[a].Count - 1 >= 2000)
{

// series[a].Count - 2000

}

as the input to my analysis methods for the count to start from.


@prosteel1

genappsforex
10 Jan 2021, 08:59

RE: RE: RE: RE: RE:

Hi Prosteel1,

We did that a few months ago but it was put aside as being counter intuitive (clients might use it wrongly)
The real problem IMO is that there is no real reason why Spotware does not give a wider access to the heart of the backtesting system (it'll only benefit the developers and the traders) as in making to & from date bot controlable, adding ticks history and enabling developers to specify how much bars / ticks should be loaded.
Our library for workarounds, overloads and extentions is getting bigger and bigger and bigger. And also the maintenance effort of it on every new release.

I just hope that not some of the brass puls the plug out of the cTrader project and puts the capacity to the proprietary system again.
 

 

 

 


@genappsforex

prosteel1
18 Jan 2021, 15:57

RE: RE: RE: RE: RE: RE:

genappsforex said:

Hi Prosteel1,

We did that a few months ago but it was put aside as being counter intuitive (clients might use it wrongly)
The real problem IMO is that there is no real reason why Spotware does not give a wider access to the heart of the backtesting system (it'll only benefit the developers and the traders) as in making to & from date bot controlable, adding ticks history and enabling developers to specify how much bars / ticks should be loaded.
Our library for workarounds, overloads and extentions is getting bigger and bigger and bigger. And also the maintenance effort of it on every new release.

I just hope that not some of the brass puls the plug out of the cTrader project and puts the capacity to the proprietary system again.
 

 

 

 

Yes true, but they are working on major updates atm. .Net Core Upgrade for example :)

I did write some code to attempt to do backtesting in the bot itself while in realtime using methods similar to above, but it wasn't successful - it's actually quite complex to code backtesting lol


@prosteel1