Backtesting broken on higher timeframes in 3.7

PanagiotisCharalampous
27 Dec 2019, 08:15

Hi mpistorius,

Can you please share the complete cBot code so that we can reproduce the issue?

Best Regards,

Panagiotis 

Join us on Telegram

mpistorius
30 Dec 2019, 03:26

RE:

PanagiotisCharalampous said:

Hi mpistorius,

Can you please share the complete cBot code so that we can reproduce the issue?

Best Regards,

Panagiotis 

Join us on Telegram

 

Hi Panagiotis,

This happens on the simplest of cBots, here is a broken example:

using System;
using System.Linq;
using cAlgo.API;
using cAlgo.API.Indicators;
using cAlgo.API.Internals;
using cAlgo.Indicators;

namespace cAlgo.Robots
{
    [Robot(TimeZone = TimeZones.UTC, AccessRights = AccessRights.None)]
    public class Test : Robot
    {
        DirectionalMovementSystem adx;
        
        protected override void OnStart()
        {
            adx = Indicators.DirectionalMovementSystem( MarketData.GetBars( TimeFrame.Daily ), 14 );    
        }

        protected override void OnTick()
        {
            Print( adx.ADX.LastValue );
        }
    }
}

Steps to reproduce:

1. Build the above cBot
2. Add an instance to a chart, using any timeframe other than m1 (say h4)
3. On the Backtesting tab, in the Backtesting Settings, change the Data field to 'h4 bars from Server (open prices)' from its default m1 bars setting
4. Run the backtest

Result: Nothing happens, error in Log window:    
Crashed in OnStart with NotSupportedException: Failed to load Daily bars for symbol EURUSD. GetBars is supported with data sources: "Tick data from Server" and "m1 data from Server"    

PanagiotisCharalampous
30 Dec 2019, 09:15

Hi mpistorius,

Thanks. The message you get explains what happens

GetBars is supported with data sources: "Tick data from Server" and "m1 data from Server"    

Best Regards,

Panagiotis 

Join us on Telegram

mpistorius
30 Dec 2019, 11:26

RE:

PanagiotisCharalampous said:

Hi mpistorius,

Thanks. The message you get explains what happens

GetBars is supported with data sources: "Tick data from Server" and "m1 data from Server"    

Best Regards,

Panagiotis 

Join us on Telegram

Hi Panagiotis,

I understood the message, but this is in my view a bug and certainly a regression from 3.6.  I am unable to backtest/optimise my bots in any reasonable amount of time in 3.7 if I'm forced to use only m1 data and had to revert back to 3.6 to continue working.  I can't use the old syntax (MarketData.GetSeries()) to replicate behavior from 3.6.  Are you saying the above is intended behavior, and you have removed the possibility to backtest on any open prices other than m1? That is a critical feature IMHO.

PanagiotisCharalampous
30 Dec 2019, 11:38

Hi mpistorius,

 I can't use the old syntax (MarketData.GetSeries()) to replicate behavior from 3.6

This is not true. We haven't removed any functionality. The below line of code should work fine in 3.7.

 adx = Indicators.DirectionalMovementSystem(MarketData.GetSeries(TimeFrame.Daily), 14);

 Best Regards,

Panagiotis 

Join us on Telegram

mpistorius
30 Dec 2019, 12:04

RE:

PanagiotisCharalampous said:

Hi mpistorius,

 I can't use the old syntax (MarketData.GetSeries()) to replicate behavior from 3.6

This is not true. We haven't removed any functionality. The below line of code should work fine in 3.7.

 adx = Indicators.DirectionalMovementSystem(MarketData.GetSeries(TimeFrame.Daily), 14);

 Best Regards,

Panagiotis 

Join us on Telegram

But that is my point.  The above line of code doesn't work in 3.7 if the backtest data setting is set to anything higher than m1, while the same code does work in 3.6.  If you follow the STR I set out earlier and run it in 3.7, you'll see the error.  No error in 3.6

mpistorius
30 Dec 2019, 12:11

RE: RE:

mpistorius said:

PanagiotisCharalampous said:

Hi mpistorius,

 I can't use the old syntax (MarketData.GetSeries()) to replicate behavior from 3.6

This is not true. We haven't removed any functionality. The below line of code should work fine in 3.7.

 adx = Indicators.DirectionalMovementSystem(MarketData.GetSeries(TimeFrame.Daily), 14);

 Best Regards,

Panagiotis 

Join us on Telegram

But that is my point.  The above line of code doesn't work in 3.7 if the backtest data setting is set to anything higher than m1, while the same code does work in 3.6.  If you follow the STR I set out earlier and run it in 3.7, you'll see the error.  No error in 3.6

To make it clear, if I change the line to what you suggest (just replace GetBars with GetSeries), I still get the same crash if I run the cBot in 3.7.  The only difference is that the code compiles with a warning that GetSeries is obsolete.  But I expect GetSeries was implemented as a wrapper around GetBars in the new build

mpistorius
07 Jan 2020, 10:48

RE: RE: RE:

mpistorius said:

mpistorius said:

PanagiotisCharalampous said:

Hi mpistorius,

 I can't use the old syntax (MarketData.GetSeries()) to replicate behavior from 3.6

This is not true. We haven't removed any functionality. The below line of code should work fine in 3.7.

 adx = Indicators.DirectionalMovementSystem(MarketData.GetSeries(TimeFrame.Daily), 14);

 Best Regards,

Panagiotis 

Join us on Telegram

But that is my point.  The above line of code doesn't work in 3.7 if the backtest data setting is set to anything higher than m1, while the same code does work in 3.6.  If you follow the STR I set out earlier and run it in 3.7, you'll see the error.  No error in 3.6

To make it clear, if I change the line to what you suggest (just replace GetBars with GetSeries), I still get the same crash if I run the cBot in 3.7.  The only difference is that the code compiles with a warning that GetSeries is obsolete.  But I expect GetSeries was implemented as a wrapper around GetBars in the new build

Hi Panagiotis,

Have you guys managed to reproduce this issue?  I get the feeling that we've been talking around each other in this thread, but I can assure you there is a bug here.  It boils down to 

MarketData.GetSeries( Timeframe.Daily );

crashing in 3.7 when used on backtesting data for bars above m1 open prices.

PanagiotisCharalampous
07 Jan 2020, 10:50

Hi mpistorius,

Yes we did and will be fixed soon.

Best Regards,

Panagiotis 

Join us on Telegram

PanagiotisCharalampous
08 Jan 2020, 08:26

Hi mpistorius,

This should have been fixed in the latest update of Spotware cTrader Beta.

Best Regards,

Panagiotis 

Join us on Telegram

mpistorius
08 Jan 2020, 13:33

RE:

PanagiotisCharalampous said:

Hi mpistorius,

This should have been fixed in the latest update of Spotware cTrader Beta.

Best Regards,

Panagiotis 

Join us on Telegram

Hi Panagiotis,

Unfortunately, it looks like we're not out of the woods yet.  It appears to be fixed for small tests cases (like backtesting 2 symbols over a month works fine), but not for larger bots.  My bot now crashes with a message

cBot crashed: Cache file `` is damaged and has been removed. It will be updated on next run. If the error persists, try clearing target symbol cache manually.

if I try to run it on more symbols, for longer ranges. (The file name is empty, I didn't modify the error message). As an example, I modified the simple minimal multi-symbol bot I posted earlier to run over the 28 major pairs and backtested it over 1 year (06/01/2019-06/01/2020) on h4 open prices.  I also tried deleting the contents of my AppData/Roaming/demo-cTrader/BacktestingChache folder and re-ran it, but I consistently get the above crash while the bot is downloading symbol data.

Please let me know if you can reproduce the issue.  Below is the modified bot that crashes on the latest 3.7 update

using System;
using System.Linq;
using System.Collections.Generic;

using cAlgo.API;
using cAlgo.API.Indicators;
using cAlgo.API.Internals;
using cAlgo.Indicators;

namespace cAlgo.Robots
{
    [Robot(TimeZone = TimeZones.UTC, AccessRights = AccessRights.None)]
    public class Test : Robot
    {
        string[] symbols = 
        {
            "AUDCAD",
            "AUDCHF",
            "AUDJPY",
            "AUDNZD",
            "AUDUSD",
            "CADCHF",
            "CADJPY",
            "CHFJPY",
            "EURAUD",
            "EURCAD",
            "EURCHF",
            "EURGBP",
            "EURJPY",
            "EURNZD",
            "EURUSD",
            "GBPAUD",
            "GBPCAD",
            "GBPCHF",
            "GBPJPY",
            "GBPNZD",
            "GBPUSD",
            "NZDCAD",
            "NZDCHF",
            "USDJPY",
            "NZDJPY",
            "NZDUSD",
            "USDCHF",
            "USDCAD"
        };
        
        List< Strategy > strategies = new List< Strategy >();
        
        protected override void OnStart()
        {
            foreach ( var sym in symbols )
            {
                var s = new Strategy();
                s.init( this, sym );
                strategies.Add( s );
            }
        }

    
    }
    
    public class Strategy 
    {
        RelativeStrengthIndex rsi;
        Robot bot;
        string symbol;
        
        public void init( Robot bot, string symbol )
        {
            this.symbol = symbol;
            this.bot = bot;
            
            Bars bars = bot.MarketData.GetBars( bot.TimeFrame, symbol );
            rsi = bot.Indicators.RelativeStrengthIndex( bars.ClosePrices, 14 );
            
            bars.BarOpened += OnBarOpened;
        }
        
        void OnBarOpened(BarOpenedEventArgs obj)
        {
            if ( rsi.Result.Last(1) > 70 && rsi.Result.Last(2) <= 70 )
            {
                bot.ExecuteMarketOrder( TradeType.Sell, symbol, 10000, symbol, 10.0, 10.0 );
            }
        }
    }
}

mpistorius
08 Jan 2020, 16:17

RE: RE:

Some more info on the issue above.  I can get around the crash if I use other timeframes, like h3 or h6, but it consistently crashes on h4, even after clearing the cache.  Unfortunately, my strategy is developed for h4.  After downloading some different timeframe data, the test progresses slightly further on h4 before crashing, but it looks to break in the same spot every time.

 ¯\_(ツ)_/¯

TOPGUNFX
14 Mar 2020, 19:58

When is the fix likely to be live? Thanks.

PanagiotisCharalampous
16 Mar 2020, 09:24

Hi TOPGUNFX,

We managed to reproduce it and we are working on a fix. We do not have an ETA yet.

Best Regards,

Panagiotis 

Join us on Telegram