JerryTrader
21 Apr 2021, 11:37

RE:

Ok, the problem is not the event themselves, it was in my code, my bad !

I thought the issue came from the Positions collection because while debugging step by step, the debugger gave me weird values.
And I already noticed that sometimes, the debugger jumps instructions, and sometimes, some values are wrong, or even not computed, ending with a message like :

Cannot obtain value of local or argument as it is not available at this instruction pointer, possibly because it has been optimized away.

I copied/pasted my class into the cBot project (and not my reusable DLL), and the debugger works fine, and helped to find what my issue was.
My guess is that when cTrader compiles the bot and its dependencies, it compiles it in Release mode, with optimization, which makes the step by step debugging almost impossible.

So, the real question of this thread is more :

How to debug with reusable DLLs ?
Is there a way to say to cTrader : do not optimize this DLL ?

I already tried setting my project properties and VS, like recommended in this post : 

Is that supposed to work ? Is it because of the way cTrader compiles dependencies or am I missing some configuration ?

Thanks a lot for your help,
Cheers,
Jerry

JerryTrader
13 Apr 2021, 15:56

RE:

Thanks Panagiotis for your complete answer. It is much more clear now.
I will have to think more about how I can develop this collection.

Would it be possible, on your side, to consider the feasibility of adding a way to find the Position given a PendingOrder (if this order has been filled, of course) ?
I don't know, something like an event that we subscribe on the PendingOrder, for example.

Or am I still misunderstanding something, and it doesn't make any sense to request such a feature ?

Anyways, a big thanks to you guys, Ahmad and Panagiotis.
By your precious support, you add a lot of value to this platform and API (that truly lacks documentation by the way).

Keep up the good work !

JerryTrader
13 Apr 2021, 12:06

RE:

amusleh said:

Why you are trying to link OnTick with PendingOrders Filled event?

I'm not trying to link them, I'm trying to create a TradeCollection class that groups some PendingOrders and Positions together.
To keep it up to date, I was relying on the PendingOrders.Filled event to remove the order from one collection, and add the new Position to the other collection.

When you look at an API and find an event like PendingOrders.Filled, you expect that it can be used to know exactly when an order is filled.
The documentation doesn't mention anywhere that this event is called sequentially, and after the OnTick (which is partly wrong, since there is a bug depending on the backtesting speed).
 

Code your cBot in a way that it doesn't rely on the sequence that these methods will be called, because in future it can change.

I wish I could. Usually, you do this by relying on events, which can't be done here.

PendingOrders Filled event is where you get the position for triggered Pending Order, but in what sequence it will be called is not guaranteed and we call it as soon as possible.

This is exactly my biggest problem here.
So how can you close a position in the OnTick, if you didn't received the PendingOrders Filled event (since it seems to be the only way to get the position for a triggered PendingOrder) ?
Relying on some text parsing in the order label is just a hack to fill a gap in the API.

If its very important for you, then you can create you own observable collections of pending orders and positions, use OnTick and Pending Orders events to keep your collections up to date based on your preference, you can create a library with your own classes and use it on your bots.

This is exactly what I do. And keeping my collection up to date is exactly what I'm trying to achieve.
But using OnTick and PendingOrders events "based on my preference" ... I wish I could but I'm facing the issues I'm trying to explain from the beginning :

- If I use OnTick, I can see that a PendingOrder is missing, but I can't find a way to get the Position yet
- If I use PendingOrders.Filled, I can get the Position for the triggered pending order, but it is too late : in the OnTick I already tried to close a pending order that doesn't exist anymore because it has been triggered.

JerryTrader
13 Apr 2021, 11:30

RE:

amusleh said:

Hi,

You can use all Linq methods and there is no difference, if there is an API method I recommend you to use it instead of Linq, otherwise use Linq.

I prefer to use the API when available, but since it is not available on the PendingOrder, for the sake of consistency, I will use Linq.

Thanks for your answer.

JerryTrader
12 Apr 2021, 11:32

Hi, 

First, I would like to apologize if you are offensed, because I feel like you're defensive.
I don't say that cTrader nor the API is bad, but in my opinion, this is a bug that needs to be fixed, and that shows some lacks from the API (using the order label might work but it is just a "hack" to workaround a lack/bug).

My only goal here is to go forward and help as much as I can.

amusleh said:

User can't change the order label or comment, and there is no reason to add a Position property to pending order, you can use the PendingOrder.Filled event.

There is no way of knowing which PendingOrder turned into which Position, except from listening to an non deterministic event that "sometimes" is called before, "sometimes" is called after the OnTick.
Without talking about the event raise bug, there is a relation between a PendingOrder and a Position, and this is already a reason to let the API user to retrieve an order from a position and vice versa.

amusleh said:

Your case is exceptional, you want to get notified as soon as possible when a pending order if filled, so that's why the PendingOrder Filled event might not be useful for you but it works fine most of the time.

I don't think that maintaining a list of PendingOrders and Positions is something exceptional.

amusleh said:

As I said, OnTick has precedence over PendingOrder filled event, we want to notify the user as soon as possible when a new tick is arrived.

I understand that. But in the other hand, if we take the example of an ObservableCollection, your collection is never modified before you get the event.
This is the only way you can be guaranteed that your collection is up to date.

JerryTrader
10 Apr 2021, 20:17

Ok, 

So there's no way, through the API, to "map" a PendingOrder to a Position, like we have in the PendingOrderFilledEventArgs class.
Thanks for the tip, using the order label or comment is feasible. It is risky to count on some text that the user can modify, though.

It looks like the API is missing something here.
Maybe you could add a "Position" property to the PendingOrder interface, that would be null if the PendingOrder is not filled, and that returns the Position when filled ?
So that, in the OnTick, we could retrieve the Position from the PendingOrder.

What do you think ?

JerryTrader
10 Apr 2021, 17:28

amusleh said:

These types of issues can happen while trading on a live market, noting is guaranteed, you should take the TradeResult after each trade operation call and check if it was successful or not.

Yes, I check the TradeResult after each trade operation call, but then ... I don't know what to do with it ...
If I want to cancel a PendingOrder that doesn't exist anymore, I receive a TechnicalError (and not an EntityNotFound as I would expect), but I have no more information about what happened, and I can't find any way to "guess" that the order just get filled, but the event has not been raised yet, and that this order has now turned to a Position (so I don't have the Position ID).

Raising OnTick has precedence over anything else, and you can also check if Bid/ask reached your pending order target price or not inside OnTick.

Ok, I could check that to anticipate the PendingOrders.Filled event (which makes the event obsolete if I have to manually check the conditions of all my PendingOrders and increase my CPU usage). Why not, if I can workaround this bug ...

But then, if I find that the Bid/Ask reached my pending order, how can I find the corresponding position ID ? I don't see any ways to find it except from waiting for the PendingOrders.Filled event. Maybe I missed it ?

JerryTrader
09 Apr 2021, 19:59

Hi amusleh, 
Thank you for your support.

If it is working in different ways depending on the backtesting speed, it is necessarily a bug.

I don't know how it is done under the hood, but in my opinion, it makes more sense to raise PendingOrders.Filled event first.
Otherwise, you end up in strange state where in the OnTick, you want to close a pending order, but this one doesn't exist anymore, because it has already been filled, but you're not yet aware of this.

I don't know how I can workaround this...

JerryTrader
07 Apr 2021, 11:11 ( Updated at: 07 Apr 2021, 11:12 )

Hi,

You should assign your result to wvf[index].
Regarding wvf * -1, it just inverts the plot.

You should be ok with :

public override void Calculate(int index)
{
    wvf[index] = -((Math.Max(Bars.ClosePrices[index], pd) - Bars.LowPrices[index]) / (Math.Max(Bars.ClosePrices[index], pd))) * 100;
}

JerryTrader
06 Apr 2021, 20:33

I couldn't agree more on that point.

If LoadMoreHistory exists, there's no reason why it should not be available for backtests.

JerryTrader
06 Apr 2021, 19:25

I think I got a similar issue due to a lack of history during backtest.

I have a complex multitimeframe indicator. 
I use it on M5 and it draws a dozen of [Output] from a referenced indicator used in M30.
Every outputs are well displayed in the cTrader Automate Indicator tab, as well as the cBot tab on a live instance, and in the cTrader Trade section.

But when I want to backtest a bot and display my indicator on the backtested chart, my outputs are not displayed before 11-Feb-21.
 

From the below code, I figured out that I'm missing data before this date.

Bars _trendFilterBars = MarketData.GetBars(TimeFrame.Minute30);

// This returns "11-Feb-21 13:30:00"
var firstMtfOpenTime = _trendFilterBars.OpenTime[0];

I guess this is a good example of why LoadMoreHistory could be useful, even on the backtest part.

JerryTrader
06 Apr 2021, 16:47

RE:

PanagiotisCharalampous said:

Hi JerryTrader,

Indicators added to the backtesting chart cannot use the backtesting log to print messages. Maybe try calling the indicator from a cBot. In this case, you should be able to see the messages.

Best Regards,

Panagiotis 

Join us on Telegram

Hmmm ok. 
I will try that. I can also log indicator messages to a file so that I don't have to create a cBot just to get logs from an indicator.

As long as I have workarounds, I'm fine ;)
But I think it could be handy to add a feature to let the user choose the log sources in the Log tab.

While you're there (sorry, this is off-topic but I can't find any answer...):
For 2-3 weeks now, I don't receive any notifications about the topics I am subscribed to. For example, if I don't visit/refresh this page, I'm not able to see if you answered to it.
I checked my settings and everything seems fine to me. I posted a message in the cTrader Web section, and I reported an issue through Pepperstone cTrader. I really have no idea where to report this so it can be fixed...

Can you please redirect me to the right place, or report this to the right team ?

JerryTrader
06 Apr 2021, 11:22

RE:

Hi PanagiotisCharalampous,

As I said, my chart does, not my cBot.

JerryTrader said:

For some unknown reason, it doesn't display the same way when I add it to a chart of a cBot backtest, so I would like to see the logs of my indicator to understand why some Outputs are not displayed on the chart.

I have a complex indicator that works fine in the Indicators tab, the cBot live instance and in cTrader Trade, but when I add it to a chart of a cBot backtest, it doesn't behave the same way.
I have no idea why an indicator would be displayed differently when used with the same settings, so I want to debug this by writing some messages and see what's going on.

Hence my question:

JerryTrader said:

Is there a way to display logs (or get a file somewhere) of an indicator when it is instanciated elsewhere than the Indicator tab?

JerryTrader
06 Apr 2021, 11:05

RE: RE:

JerryTrader said:

NOK : cTrader Automate section -> cBot -> Basic cBot that does nothing else than printing a message on start

using cAlgo.API;

namespace cAlgo
{
    [Robot(TimeZone = TimeZones.UTC, AccessRights = AccessRights.FileSystem)]
    public class PrintMessagesTests : Robot
    {
        protected override void OnStart()
        {
            base.OnStart();
            Print("Print Messages test bot started");
        }

        protected override void OnStop()
        {
            base.OnStop();
            Print("Print Messages test bot stopped");
        }
    }
}

JerryTrader
06 Apr 2021, 10:54

Hi, 

You can use DateTime.Now to know the current time.

Then you can easily check if your datetime.Hour is in the range you want.

Something like this (not tested though):

protected override void OnBar()
{
    int index = Bars.Count - 2;

    var now = DateTime.Now;
    if ((now.Hour > 19 && now.Hour < 21) ||
        (now.Hour > 2 && now.Hour < 4) ||
        (now.Hour > 8 && now.Hour < 10))
    {
        Entry(index);
    }

    Exit(index);
}

JerryTrader
06 Apr 2021, 10:38 ( Updated at: 21 Dec 2023, 09:22 )

RE:

PanagiotisCharalampous said:

The indicator should print the messages if you are in the cTrader Automate section.

I can't get the indicator logs displayed anywhere else than the Indicator instance itself.

If not, please share the source code to check.

using System;
using cAlgo.API;
using cAlgo.API.Internals;
using cAlgo.API.Indicators;
using cAlgo.Indicators;

namespace cAlgo
{
    [Indicator(IsOverlay = true, TimeZone = TimeZones.UTC, AccessRights = AccessRights.None)]
    public class TestPrintMessages : Indicator
    {
        [Parameter(DefaultValue = 0.0)]
        public double Parameter { get; set; }
        
        [Output("Main")]
        public IndicatorDataSeries Result { get; set; }

        [Output("Moving Average")]
        public IndicatorDataSeries ResultMtfMa { get; set; }

        private Bars _mtfBars;
        private MovingAverage _mtfMa;
        
        protected override void Initialize()
        {
            _mtfBars = MarketData.GetBars(TimeFrame.Daily);
            _mtfMa = Indicators.MovingAverage(_mtfBars.ClosePrices, 100, MovingAverageType.Exponential);
        }

        public override void Calculate(int index)
        {
            var mtfMaIndex = _mtfBars.OpenTimes.GetIndexByTime(Bars.OpenTimes[index]);
            if (mtfMaIndex != -1)
            {
                Print(string.Format("Index is {0}. Mtf Index is {1}", index, mtfMaIndex)); 
                ResultMtfMa[index] = _mtfMa.Result[mtfMaIndex];
            }
        }
    }
}

Given the code above :

OK : cTrader Automate section -> Indicators -> Test Print Messages instance

NOK : cTrader Automate section -> cBot -> Basic cBot that does nothing else than printing a message on start

NOK : cTrader Automate section -> cBot -> Backtesting the basic cBot that does nothing else than printing a message on start

JerryTrader
30 Mar 2021, 12:28

RE: RE: RE:

amusleh said:

You can use properties and events, but some times they don't behave the way they should, so that's why I said not use properties when referencing custom indicators on a cBot.

Interesting ! So I guess this is a bug.
Did you find under which conditions they don't work as expected ? Just so I can try to workaround this.

Thanks Ahmad !

JerryTrader
30 Mar 2021, 00:56

RE:

amusleh said:

You have to pass all indicator parameters a value on "GetIndicator" method, otherwise it will not work, and you can only use the indicator outputs not properties.

Why are you saying not to use properties ? In some of my bots and indicators, I use properties and events, and it seems to work fine.
I double checked the documentation, but no mentions saying if we should or shouldn't use properties.

I read in a topic (but can't find it back) that an indicator needs to be calculated (by accessing an output for example) before being able to access a property.

Can you please detail your answer a bit more ?

Thanks !

JerryTrader
26 Mar 2021, 09:21

RE:

amusleh said:

Hi,

BarOpened/Tick events were added in recent versions of Automate API, they were not available in older versions.

Both do the same thing but if you are developing a single time frame/symbol based indicator/cBot then you should use the overridden methods because that's much easier and you don't have to subscribe for the events, and if you are developing a multi symbol/ time frame indicator/cBot then you should use the BarOpened/Tick events.

Ok, thanks for your detailed and precise answer !

Everything's clear

JerryTrader
17 Mar 2021, 16:15 ( Updated at: 17 Mar 2021, 16:20 )

RE: RE:

JerryTrader said:

I will think of a workaround, and post it there if I find some.

Ok, that was an easy one ...

For those who are interested in doing similar things, here's how I did it (I did not test this code, so it might not compile as is).

Let's say I want to draw a cloud between MA(100) and MA(200), and this cloud has to be green sometimes, and red other times.

[Cloud(MA100RedCloud, MA200RedCloud, FirstColor = "Red", Opacity = 0.2)]
[Cloud(MA100GreenCloud, MA200GreenCloud, FirstColor = "Green", Opacity = 0.2)]
[Indicator(IsOverlay = true, TimeZone = TimeZones.UTC, AccessRights = AccessRights.None)]
public partial class CloudIndicator : Indicator
{
  [Output("MA100", LineColor = "Blue")]
  public IndicatorDataSeries OutMa100Red { get; set; }

  [Output("MA200", LineColor = "Yellow")]
  public IndicatorDataSeries OutMa200Red { get; set; }


  #region Invisible Lines to draw clouds
  [Output("MA100_RedCloud", LineColor = "Transparent")]
  public IndicatorDataSeries OutMa100Red { get; set; }

  [Output("MA200_RedCloud", LineColor = "Transparent")]
  public IndicatorDataSeries OutMa200Red { get; set; }

  [Output("MA100_GreenCloud", LineColor = "Transparent")]
  public IndicatorDataSeries OutMa100Green { get; set; }

  [Output("MA200_GreenCloud", LineColor = "Transparent")]
  public IndicatorDataSeries OutMa200Green { get; set; }
  #endregion

  public override void Calculate(int index)
  {
    // Calculate your MA100 / MA200
    // Then set outputs for green or red cloud depending on what you want
  }
}

I hope this is clear,
Let me know if you want a full sample

Cheers,
Jerry