Bad data when backtesting

Uche
06 Aug 2017, 19:59

RE:

lec0456 said:

I am backtesting an algorithm. I backtested using 2 different brokers.  IC Market's data seems to be really messed up. 

http://icmarkets.ctrader.com/c/n2Hpn

for example chucks of data is missing. On October 15 2015 the data jumps from 5:45 to 9:45.  this trows off my indicators and algorithm.  there are also negetive spreads that appear as large as -5 or more pips.  This is just inaccurate. 

Is there an explaination or a way to fix this?  

It seems the data is just currupted.

Which means backtesting on IC amrkets data is worthless. What good is testing an algorithm  using tick data that is inaccurate?

Could spotware please help?

Some brokerages do have better data and always do a data check with your algorithm before executing trades,best way is to create a seperate data-checking class that you can call every tick or every 1 minute.

lec0456
06 Aug 2017, 22:21

How do you do that? How can I check? And if its bad what can I do?

Uche
07 Aug 2017, 10:22

RE:

lec0456 said:

How do you do that? How can I check? And if its bad what can I do?

Create a logic that flags a boolean if data is inconsistent or unusual and refreshdata/skip trade executions if detected.Example;

High should always be greater than or equal to low,time lag in data should be minimal,Ask price should normally be greater than or equal to Bid price except if the negative spread is reasonable and tradable;most times large negative spreads are inverted prices created in error.

From experience,the major brokerages using agregated pricing generates these errors in tick data and makes computing signals for scalping strategies problematic,using the data check  in your algorithm will cause a reduced number of backtest trades in affected brokerages;this should give you an idea of the broker's suitability for your strategy.

I suggest you use a flag and hold method for trade signals, can help improve execution in some cases if your algorithm depends on data from Higher Timeframes.