cAlgo prices/ticks wrong on backtesting ?

albeottt
18 Jun 2016, 08:30

Screenshot links:

https://s31.postimg.org/n0a86gwaz/c_Algo1.png

https://s31.postimg.org/49cwa1r4b/c_Algo2.png

albeottt
18 Jun 2016, 23:30

That was it! Huge negative spreads everywhere. The thing that bugs me now is why there's no way to show the bid AND ask lines or ticks on the backtesting screens. This would help to see this issue immediately.

I had to fix the line on the bot that rounds, and added a counter for positive and negative spread totals. Any recent data of the past months is clean with no negatives. Going back a year, suddenly there's huge amounts of negatives. This is on Pepperstone. Which broker should I use, that will provide clean data for at least a year? Do I have to wipe the cTrader folders to clear out the bad tick data?

Fixed code for anyone else to investigate if you're getting clean ticks:

using System;
using System.Linq;
using cAlgo.API;
using cAlgo.API.Indicators;
using cAlgo.API.Internals;
using cAlgo.Indicators;

namespace cAlgo
{
    [Robot(TimeZone = TimeZones.UTC, AccessRights = AccessRights.None)]
    public class Spread : Robot
    {


        int pos = 0;
        int neg = 0;

        protected override void OnTick()
        {
            if (Symbol.Spread > 0)
            {
                Print("Spread = " + Math.Round((Symbol.Spread / Symbol.PipSize), 2));
                //Print("Spread = " + Symbol.Spread);
                //Print("Pipsize = " + Symbol.PipSize);
                pos++;
            }
            if (Symbol.Spread < 0)
            {
                Print("Spread = " + Math.Round((Symbol.Spread / Symbol.PipSize), 2));
                //Print("Spread = " + Symbol.Spread);
                //Print("Pipsize = " + Symbol.PipSize);
                neg++;
            }
        }

        protected override void OnStop()
        {
            Print("Pos: " + pos + ", Neg: " + neg);
        }
    }
}

Spotware
22 Jun 2016, 12:45

Dear Traders,

As you most probably already know, Sell positions open on bid prices and close on ask prices. Buy positions open on ask prices and close on bid prices. The bid and ask prices are defined by the Liquidity Providers. Usually the ask price is higher than the bid price. Sometimes the ask price may become lower than the bid price based on the liquidity provided by the LPs. Thus, the spread could be negative. 

The candles on the charts are created based on the bid prices. It's possible for Sell positions to be closed on a price which is not located on the bar at that time, since sell positions close on ask prices. We kindly ask you to contact your Brokers regarding any execution questions you may have.

albeottt
02 Mar 2017, 11:56

RE:

Spotware said:

Dear Traders,

...

Hello Spotware,

We need clarification on this issue. These ticks are NOT normal quotes. They have never existed, and some broker's cAlgo tick data have them, along with Spotware demo server. Please try the following cBot, and then view the log:

http://pastebin.com/EzQYLKae

Tick data backtest on Spotware cAlgo Demo server for USDJPY period 11/07/2015 to 19/07/2015:

- Total ticks: 413869, Positive: 381424, Negative: 32445, Spreads -0.4 to -25.0 comprise 6.77074 % of tick data
...
- Spread of -10.5 pips appears 37 times
- Spread of -10.4 pips appears 43 times
- Spread of -10.3 pips appears 41 times
...
- Spread of -9.5 pips appears 51 times
...

GBPUSD 02/07/2015 to 08/07/2015:

- Total ticks: 1221719, Positive: 1101584, Negative: 120135, Spreads -0.4 to -25.0 comprise 8.81799 % of tick data

On other broker's servers, these periods are months long and go negative dozens of pips. Pepperstone and IC Markets for example: most majors before August 14, 2015 go negative spread for months, while everything after Aug 15 2015 is perfect for the few symbols I've tried. I might say that MOST symbols from most common brokers utilizing cAlgo may not have a time period longer than 1.5 years, without a period of faulty ticks.

The only way around this is to program your bot to not trade these problematic time periods, as many systems backtesting either crash immediately or make a million dollars once these periods with large negative-spread ticks are encountered. This isn't the solution to the problem.

What a normal time period looks like:

- Total ticks: 8085634, Positive: 8085634, Negative: 0, Spreads -0.4 to -25.0 comprise 0 % of tick data
- Spread of 0.1 pips appears 111994 times
- Spread of 0.2 pips appears 206997 times
- Spread of 0.3 pips appears 581118 times
- Spread of 0.4 pips appears 1267433 times

Regards

ctid299233
02 Mar 2017, 17:35

Clean tick data anyone?

Hi @Spotware,

I found the same, several days with tick data repeated within the same minute even 50 times and backtests takes all these trades, resulting in very inacurate results.

It's just a matter of cleaning those days out.

Does anyone have clean tick data, maybe in CSV, maybe from another provider?

Thank you!