Missing bars

gary_forex
24 Jul 2023, 13:51 ( Updated at: 24 Jul 2023, 13:52 )

Missing Bars Indicator

I have created the following quick & dirty indicator to further explore this issue.

I believe some of the instances may be caused by zero trades, as per the 1 minute chart AUSUSD around 07:00 UTC most days.

These appear as 1px high candles on the cTrader chart. But don't have a corresponding Bars[index].

using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Text;
using cAlgo.API;
using cAlgo.API.Collections;
using cAlgo.API.Indicators;
using cAlgo.API.Internals;

namespace cAlgo
{
    [Indicator(AccessRights = AccessRights.None)]
    public class MissingBars : Indicator
    {
        [Parameter("Debugger", DefaultValue = false, Group = "Debugging")]
        public bool Debugger { get; set; }

        [Output("Missing Bars", IsHistogram = true, LineColor = "5e4fa2", PlotType = PlotType.Histogram, Thickness = 8)]
        public IndicatorDataSeries Missing { get; set; }

        protected override void Initialize()
        {
            if (Debugger) System.Diagnostics.Debugger.Launch();
            
            // get internal uint Bars.TimeFrame.Size
            var property = Bars.TimeFrame.GetType().GetProperty("Size", BindingFlags.NonPublic | BindingFlags.Instance);
            uint _BarMinutes = (uint)property.GetValue(Bars.TimeFrame);

            Print("_BarMinutes = {0}", _BarMinutes);

            
            DateTime _OpenTime = Bars[0].OpenTime;
            
            for (int i=0; i<Bars.Count; i++) {
                if (Bars[i].OpenTime != _OpenTime) {
                    Print("Missing bar: Expected {0:ddd d MMM yyyy HH:mm} but was {1:ddd d MMM yyyy HH:mm} instead.", _OpenTime, Bars[i].OpenTime);
                    Missing[i] = 1;
                    _OpenTime = Bars[i].OpenTime; // get back on track
                }
            
                _OpenTime = _OpenTime.AddMinutes(_BarMinutes);

                // March 2023 seems to break this Apr-Oct/Nov-Mar DST rule
                if (_OpenTime.DayOfWeek == DayOfWeek.Friday && _OpenTime.Hour == ((_OpenTime.Month >= 4 && _OpenTime.Month <= 10) ? 21 : 22) && _OpenTime.Minute == 0) {
                    // jump weekend close
                    _OpenTime = _OpenTime.AddMinutes(2 * 24 * 60);
                    //Print("Jumped weeked to {0:ddd d MMM yyyy HH:mm}", _OpenTime);
                    if (Bars[i + 1].OpenTime != _OpenTime) Print("Weekend jump incorrect: Expected {0:ddd d MMM yyyy HH:mm} was {1:ddd d MMM yyyy HH:mm} instead.", _OpenTime, Bars[i + 1].OpenTime);                    
                    _OpenTime = Bars[i + 1].OpenTime; // get back on track
                }
            }
        }

        public override void Calculate(int index)
        {
        }
    }
}

PanagiotisChar
25 Jul 2023, 06:22

Hi there,

If there are missing bars in your feed, talk to your broker. The price feed is a responsibility of the broker.

Aieden Technologies

Need help? Join us on Telegram

gary_forex
25 Jul 2023, 15:40

RE: Missing bars

PanagiotisChar said: 

Hi there,

If there are missing bars in your feed, talk to your broker. The price feed is a responsibility of the broker.

Have registered a new account with Pepperstone and it appears to be the same data with the same holes.

IMO just as cTrader itself fills in the blanks on the chart, it should be providing the same null values to cbots and indicators for consistency.

As end consumers we should be able to assume that Bars[i] and Bars[i+5*24*60] are 1 trading week apart on any 1 minute forex chart.

Same should also hold true over irregular closed market holidays too, though I'd be fine with having to set a parameter for that kind of fill.

PanagiotisChar
26 Jul 2023, 05:34

Hi there,

This is the convention on which the platform was built and thousands of cBots developed around it. Such a change would break thousands of projects, so realistically you cannot expect it to change. You need to develop your own project around this.

Aieden Technologies

Need help? Join us on Telegram

gary_forex
26 Jul 2023, 08:08

RE: Missing bars

PanagiotisChar said: 

Hi there,

This is the convention on which the platform was built and thousands of cBots developed around it. Such a change would break thousands of projects, so realistically you cannot expect it to change. You need to develop your own project around this.

To “not have missing data” is not a breaking change, lol, the existing indicators & bots are likely not even aware they have the potential to miss anything.. and as long as there is a pretty line on the screen, the trader is no wiser, I guess.

Anyway, I've coded around it, but it should still be addressed at some point…

The lack of any cTrader reps around here is concerning too, considering the nature of what these bots do.

PanagiotisChar
26 Jul 2023, 13:00

RE: RE: Missing bars

gary_forex said: 

PanagiotisChar said: 

Hi there,

This is the convention on which the platform was built and thousands of cBots developed around it. Such a change would break thousands of projects, so realistically you cannot expect it to change. You need to develop your own project around this.

To “not have missing data” is not a breaking change, lol, the existing indicators & bots are likely not even aware they have the potential to miss anything.. and as long as there is a pretty line on the screen, the trader is no wiser, I guess.

Anyway, I've coded around it, but it should still be addressed at some point…

The lack of any cTrader reps around here is concerning too, considering the nature of what these bots do.

It would be nice of you to show some respect to people that have more experience than you on the subject and try to help you, instead of throwing lols around. To put it in your tone and style, what you propose is nonsense and will never get implemented. No official rep will bother.

gary_forex
27 Jul 2023, 02:44

RE: RE: RE: Missing bars

It would be nice of you to show some respect to people that have more experience than you on the subject and try to help you, instead of throwing lols around. To put it in your tone and style, what you propose is nonsense and will never get implemented. No official rep will bother.

Lol, respectfully, as I expressed previously, cTrader itself, IS RENDERING THE MISSING BARS… so… this data IS IMPORTANT, or the chart would just have missing chunks. If it is important for the chart (unquestionably it is), it's important enough to provide the same data to the indicators being calculated for THAT chart…

OMG, you think I am being disrespectful?! I am reporting a bug!! Maybe you aren't a real software developer…. or can't debug properly… at any rate it seems like you're happy for your indicators & bots to consume inconsistent data… garbage in garbage out.