Strange position counting error ..

GoldnOil750
22 Jan 2016, 20:15 ( Updated at: 21 Dec 2023, 09:20 )

RE:

Hi,

are you calling the code on method "OnBar" or "OnTick" ??   usually calling this code in method "OnBar" will wait for the next bar to update it.

///S.Khan

ChasBrownTH said:

The following code is meant to display the count of currently open positions, but it often does not work.

             Positions.FindAll(Label);
            int totalPositions = Positions.Count;
            int buyPositionsCount = PositionsCount(TradeType.Buy);
            int sellPositionsCount = PositionsCount(TradeType.Sell);

             strStats += string.Format("\nPositions: {0}  Buys: {1}  Sells: {2}", totalPositions, buyPositionsCount, sellPositionsCount);

            ChartObjects.DrawText("stats", strStats, StaticPosition.TopLeft);

Results are displayed, often they are true, but not always. Please see the screenshot.

Did I forget to 'reset' or properly initialize something important ?

Many thanks to anybody who can help me figure out the cause.

ChasBrownTH
22 Jan 2016, 21:25 ( Updated at: 21 Dec 2023, 09:20 )

RE: RE:

Thanks for the suggestion, but I am calling it every 2 seconds, with a timer, also every minute to write it into mySQL .. both versions get 'crazy' responses sometimes  ;(

GoldnOil750 said:

Hi,

are you calling the code on method "OnBar" or "OnTick" ??   usually calling this code in method "OnBar" will wait for the next bar to update it.

 

///S.Khan

 

ChasBrownTH said:

The following code is meant to display the count of currently open positions, but it often does not work.

             Positions.FindAll(Label);
            int totalPositions = Positions.Count;
            int buyPositionsCount = PositionsCount(TradeType.Buy);
            int sellPositionsCount = PositionsCount(TradeType.Sell);

             strStats += string.Format("\nPositions: {0}  Buys: {1}  Sells: {2}", totalPositions, buyPositionsCount, sellPositionsCount);

            ChartObjects.DrawText("stats", strStats, StaticPosition.TopLeft);

Results are displayed, often they are true, but not always. Please see the screenshot.

Did I forget to 'reset' or properly initialize something important ?

Many thanks to anybody who can help me figure out the cause.

 

ChasBrownTH
22 Jan 2016, 21:57

Sorry, wish I could edit my earlier message, but that seems impossible ?

I should have mentioned that the mySQL history proves cAlgo is returning 'Nonsense' for HOURS, long after it was true ;(

I have instances with no open trades, but minute by minute 'reports' claiming there are trades Open .. 

It baffles me .. can anybody HELP please ?  Hello Spotware, is anybody there ?  ;)

Spotware
25 Jan 2016, 02:14

Dear Trader,

If you believe that there is an issue with any of our methods, could you please send us the simplest code that reproduces it?

You can post your code here in this thread or send it to troubleshooting@spotware.com, where it will be used only for troubleshooting purposes.

ChasBrownTH
25 Jan 2016, 09:24 ( Updated at: 21 Dec 2023, 09:20 )

RE:

I don't seem to be able to reply to Spotware's message on this page, so I am replying to myself.

Hopefully there is nothing wrong with the cAlgo methods, I already said that I may be calling the method wrongly. But in that case what am I doing wrong ?

I already posted the code I am using above, it is simple but clearly 'defective', in which case how is it wrong? It is repeated here:-

            Positions.FindAll(Label);
            int totalPositions = Positions.Count;
            int buyPositionsCount = PositionsCount(TradeType.Buy);
            int sellPositionsCount = PositionsCount(TradeType.Sell);

            strStats += string.Format("\nPositions: {0}  Buys: {1}  Sells: {2}", totalPositions, buyPositionsCount, sellPositionsCount);

            ChartObjects.DrawText("stats", strStats, StaticPosition.TopLeft);

I posted a picture, repeated here.

The information coming in from the Positions.Count; is nonsense, it has produced hours and hours of nonsense on many different machines over a period of days & weeks, but I did not realize at first since I assumed that the method could not be at fault. Instead I looked for the 'obvious error' in my own code ? Which nobody seems able to pinpoint.

Therefore I will send a copy of my Robot to troubleshooting@spotware.com  and ask that they run their own tests on it - Many thanks!

ChasBrownTH said:

Sorry, wish I could edit my earlier message, but that seems impossible ?

I should have mentioned that the mySQL history proves cAlgo is returning 'Nonsense' for HOURS, long after it was true ;(

I have instances with no open trades, but minute by minute 'reports' claiming there are trades Open .. 

It baffles me .. can anybody HELP please ?  Hello Spotware, is anybody there ?  ;)

Spotware
25 Jan 2016, 22:38

Dear Trader,

We would like to inform you that we do not provide coding assistance services. We more than glad to assist you with specific questions about cAlgo.API. You can contact one of our Partners or post a job in Development Jobs section for further coding assistance.

ChasBrownTH
26 Jan 2016, 08:33

RE: RE:

ChasBrownTH says:

Once again Spotware makes it impossible to directly answer their comments, so I will 'talk to myself', which seems to be the only solution available.

>> Spotware said:
>>    Dear Trader,
>>    We would like to inform you that we do not provide coding assistance services.

I see, so you do not want the actual Robot, but 'the simplest code that reproduces it'? 

So now I need to write a 'partial' Robot that 'fails' so that you can more easily see what may be wrong with cAlgo?

An interesting idea! So I will try to do that, since I cannot make progress with my robot until this problem is fixed.

>> Spotware said:
>>    We more than glad to assist you with specific questions about cAlgo.API. 
I take it from your lack of suggestions that there is nothing clearly and obviously wrong in the way I am already calling the method?

I believe I have asked a valid question already. HOW do I call the Positions.Count method? So far I have not received an answer to that very simple question.

>> Spotware said:
>>    You can contact one of our Partners 
>>    or post a job in Development Jobs section for further coding assistance.
I have worked with two 'Listed Developers' during the past 9 months, and paid for help, but so far neither was able to identify the cause of this problem. 

But it most definitely exists!

ChasBrownTH
27 Jan 2016, 06:47

Thanks Lucian

That looks promising, I am hoping it will 'initialize' the Positions counter. Fingers crossed, but I need a few days to confirm it has worked. The error is not continuous, it often shows up later in the week, when the market gets frantic and orders are opening and closing at high speed.

For test purposes I have left the old code and put the new code alongside, so I can compare two sets of output.

I think I have the same problem in these two code fragments, can you suggest how I might modify them please ?

         protected int PositionsCount(TradeType tradeType)
        {
            return Positions.FindAll(Label, Symbol, tradeType).Length;
        }

The above sometimes fails to return correct results, but I am unsure how to fix it.

         protected void CheckStopLossAndTakeProfit()
        {
            foreach (Position position in Positions.FindAll(Label, Symbol))
            {
                //RefreshData();
                if (!position.StopLoss.HasValue)
                {
                    SetStopLoss(position);
                }
                if (!position.TakeProfit.HasValue)
                {
                    SetTakeProfit(position);
                }
            }
        }

I think this code works, it is slightly different. I did not write either of them, only the code in my original question is mine.

Many many thanks for replying Lucian, I am very grateful for your help, this has been driving me Nuts for many months  ;)

ChasBrownTH
27 Jan 2016, 07:02

Oooops, I spoke too soon !

I tried to use:

            var positions = Positions.FindAll(Label);
            int totalPositions = positions.Count;

But found that variation will not build.

  Error CS0428: Cannot convert method group 'Count' to non-delegate type 'int'. Did you intend to invoke the method?

ChasBrownTH
27 Jan 2016, 07:05

SORRY for so many replies, I am learning as I go along  ;)

I added parentheses to positions.Count() and it now builds !

            var positions= Positions.FindAll(Label);
            int totalPositions = positions.Count();

At least it builds, I still do not know if it produces an accurate count.

ChasBrownTH
27 Jan 2016, 15:31

RE:

Thank you again Lucian, I have added those changes and they certainly work, but as before I will not not be sure if it fixes the problem until after the 'old' code produces problems, which hopefully the new code will not. I am VERY grateful to you for your time and effort. I can understand the logic you have introduced, so maybe I can learn to write better C# in the future.

Lucian said:

try:

int totalPositions= positions.Lenght;

and:

 private int PositionsCount(TradeType tradeType)

{
            var count = 0;
            foreach (var position in Positions)
            {
                if (position.Label == Label && position.SymbolCode == Symbol.Code)
                {
                    if (position.TradeType == tradeType)
                        count++;
                }            
            }
            return count;
 }

ChasBrownTH
28 Jan 2016, 16:19 ( Updated at: 21 Dec 2023, 09:20 )

RE:

Latest update: this just keeps getting stranger .. I can now see, thanks to the mySQL 'history' I am keeping, that it is not simply the Positions.Count() method that is failing, but also the entire routine that calls it, sometimes. I always run many virtual instances of the same Robot, some on remote VPS systems, geographically separate, precisely to look for such glitches. While some run smoothly others exhibit 'weird' behaviour, such as I described in the original posting.

However I have now identified a second set of problems, whereby in some cases some segments of the Robot stop working, but other segments continue. It is all very odd.

Thirdly, sometimes when I add a newer version of a Robot the older version keeps running 'in the background', even though I deleted it. I can sometimes see output coming from both versions, because I Log most key activities into mySQL and include the Version as well as the data. So then I have to completely Stop cAlgo, then reload it with only the newer version, or it keeps a 'shadow' copy, presumably somewhere in memory. It is very strange to see when I have only one instance of cAlgo running, but it is giving Dual output, including output from a deleted Robot, that was first stopped. I repeat the old Robot was DELETED! But it is still active, somewhere in 'never never land' ...

I have pages and pages of mySQL tables if anybody wants to see proof! Unfortunately I cannot send 'a stripped down sample' to Spotware so easily. Though I am trying, but they seem totally unwilling to even consider the possibility that any of this is happening, which is very disturbing. I also tried explaining it and demonstrating it to 'Consultants' with no success.

As always if anybody has any suggestions then I would be very grateful to hear them. If any reader wants to see 'proof' please contact me -> charles AT angloam DOT net

ChasBrownTH
28 Jan 2016, 18:23 ( Updated at: 21 Dec 2023, 09:20 )

RE: RE:

This is getting too ridiculous for words, so here are some pictures to clarify the problems .. note that the server was STOPPED at 15:48, the 'RED' Go changed to 'BLUE'. By all logic it ought to be OFF, not running, doing nothing at all .. quite by chance I checked the mySQL tables for an earlier issue and noticed that IT WAS STILL RUNNING !! In so far as some of the code was still actively sending mySQL table updates, five minutes AFTER I stopped the Robot. Also please note that not one but TWO copies were 'GHOST' running, notice that one has a MaxLT/MLT of 4 and the other an MLT of 20. [ MLT = max losing trades ] but the Robot simply cannot have two values for the same variable at the same time.

UPDATE: half an hour later it is still running. clearly the only way to stop it is to close cAlgo.   So every time we update our Robot we need a 'clean re-start' ?

ChasBrownTH
28 Jan 2016, 19:46

Conclusion: I now think some of the odd behaviour is because the Robot uses two timers, one of which is not being stopped at shut-down. It seems that it continues running in the background, divorced from its original environment, hence the nonsensical Positions.Count() results, they are coming from 'thin air', driven by a Timer that ought to have stopped, but didn't.

The first timer is simple:

            // start first timer with 2 second interval
            Timer.Start(2);

So in 'OnStop' I have added a Stop instruction.

         protected override void OnStop()
        {
            Timer.Stop();

        }

The second timer is not so simple, it uses 'fancy' code not written by me.

             // setup extra timer to invoke tasks each minute - checks environment etc
            System.Timers.Timer OneMinuteTimer = new System.Timers.Timer();
            OneMinuteTimer.Elapsed += new ElapsedEventHandler(OnMinuteTimer);
            OneMinuteTimer.Interval = 60000;
            OneMinuteTimer.Enabled = true;

It works very well, too well, in fact it never stops!

Has anybody got an idea how to correctly stop it ?

             OneMinuteTimer.Enabled = false;

Seems like the obvious way, but does not Build because it doesn't exist in the OnStop() context.

I am no C# expert, but I suspect it is running in a separate thread?

So can anybody tell me how do to stop it? It survives stopping the parent Robot, sometimes it even survives stopping and re-starting cAlgo itself.

Spotware
29 Jan 2016, 01:47

Dear Trader,

As said in our feedback channel, could you please provide us with the reproducing steps of the issue you have described?

Any additional information that helps us identify an issue is highly appreciated.

ChasBrownTH
29 Jan 2016, 06:02

RE:

Hello Spotware. Thank you for replying. I already sent you my Robot for you to look at, but I can well understand that you do not want that, only a 'simple example', which unfortunately is often not at all easy to create. However I persevered and worked it out myself.  ;)

After carefully analysing many hours of mySQL logs I realized that the root of the problem was with internal timers in my Robot. One of them was not stopping, even when the parent Robot was stopped that timer kept running, but divorced from its origin and source of valid data. That explained the 'nonsense' Postions.Count() data I was getting, it came from an an almost 'dead' thread, left running within the old shell of my Robot. 

I believe that stopping a Robot in cAlgo is not closing all activities it opened, but perhaps the programmer ought to do that?

Meanwhile I have tried to prevent the Timer problem by simply removing it, instead I now drive the one minute events using the 1 minute OnBar() method, which seems to work. But I think the wider issue needs to be resolved. My 'temporary solution' works for me, BUT it is quite inconvenient having to use a 1min chart to drive my Robot's housekeeping functions, I prefer to look at a 1hr timeframe normally, for a clearer overview.  I would be grateful to be taught how to stop the Timer correctly.

How to see it happening:

To reproduce the Timer problem, I believe that adding the timer creation code, listed below, into OnStart(), then have it call a private function such as writing to a file or mySQL will give the strange results I have seen. You should see that when the Robot is stopped that Timer keeps running, since it never gets stopped within the Robot.

// setup extra timer to invoke tasks each minute - checks environment etc

            System.Timers.Timer OneMinuteTimer = new System.Timers.Timer();

            OneMinuteTimer.Elapsed += new ElapsedEventHandler(OnMinuteTimer);

            OneMinuteTimer.Interval = 60000;

            OneMinuteTimer.Enabled = true;

// this Timer was created within OnStart()

In my Robot 'OnMinuteTimer' did 'housekeeping, including writing an 'Alive' status report to mySQL.

        /// <summary>

        /// every minute do housekeeping tasks

        /// </summary>

        /// OnMinuteTimer()

    public void OnMinuteTimer()

    {

        // do various regular housekeeping tasks

        SQLInsertAliveInfo

    }

/// <summary>

        /// write 'alive info' to SQL table

        /// </summary>

        /// SQLInsertAliveInfo()

        public void SQLInsertAliveInfo()

        {

            // Current UNIX timestamp

            Int32 UNIXStamp = (Int32)(DateTime.UtcNow.Subtract(new DateTime(1970, 1, 1))).TotalSeconds;

            double spread = ((Symbol.Ask - Symbol.Bid) / Symbol.PipSize);

            double volatility = 0;

            Positions.FindAll(Label);

            int totalPositions = Positions.Count;

            int buyPositionsCount = PositionsCount(TradeType.Buy);

            int sellPositionsCount = PositionsCount(TradeType.Sell);

            if (!IsBacktesting)

            {

                // add volatility report        

                minuteSeries = MarketData.GetSeries(Symbol, TimeFrame.Minute);

                double lastHigh = (minuteSeries.High.LastValue);

                double lastLow = (minuteSeries.Low.LastValue);

                volatility = ((lastHigh - lastLow) / Symbol.PipSize);

            }

            // mysql query

            // string query = "INSERT INTO alive (UNIXtime, time, date, balance, equity, spread, volatility, instance, accountid, botversion, stoploss, takeprofit, multiplier, maxlt, buys, sells) VALUES('" + UNIXStamp + "','" + Server.Time.ToString("HH:mm:ss") + "','" + Server.Time.Date.ToString("dd/MM/yyyy") + "','" + Account.Balance + "','" + Account.Equity + "','" + spread.ToString("0.00") + "','" + volatility.ToString("0.00") + "','" + Instance + "','" + Account.Number + "','" + BotVersion.Substring(7, 5) + "','" + ParamStopLoss.ToString("0") + "','" + ParamTakeProfit.ToString("0") + "','" + ParamVolumeMultiplier.ToString("0.0") + "','" + ParamMaxLosingTrades.ToString("0") + "','" + buyPositionsCount.ToString("0") + "','" + sellPositionsCount.ToString("0") + "')";

            string query = "INSERT INTO alive (UNIXtime, time, date, balance, equity, spread, volatility, instance, accountid, botversion, stoploss, takeprofit, multiplier, maxlt, orders, buys, sells) VALUES('" + UNIXStamp + "','" + Server.Time.ToString("HH:mm:ss") + "','" + Server.Time.Date.ToString("dd/MM/yyyy") + "','" + Account.Balance + "','" + Account.Equity + "','" + spread.ToString("0.00") + "','" + volatility.ToString("0.00") + "','" + Instance + "','" + Account.Number + "','" + BotVersion.Substring(7, 5) + "','" + ParamStopLoss.ToString("0") + "','" + ParamTakeProfit.ToString("0") + "','" + ParamVolumeMultiplier.ToString("0.0") + "','" + ParamMaxLosingTrades.ToString("0") + "','" + totalPositions.ToString("0") + "','" + buyPositionsCount.ToString("0") + "','" + sellPositionsCount.ToString("0") + "')";

            Print(query);

            //create command and assign the query and connection from the constructor

            MySqlCommand cmd = new MySqlCommand(query, connection);

            //Execute command

            cmd.ExecuteNonQuery();

        }

ClickAlgo
31 Jan 2016, 17:25

Hello my friend,

declare this as a private variable in the class

System.Timers.Timer OneMinuteTimer = new System.Timers.Timer();

 protected override void OnStop()
{
     OneMinuteTimer.Enabled = false;
}

Now you can use the private variable in the scope of the class to stop it, even the OnStop event class of cAlgo, you are correct this will stop the timer and it is not threaded. There could be a possibility that when you stop the robot the instance is still in memory. The robot instance should really unload from memory when you stop the instance in cAlgo. If you feel that this is not the case I would provide Spotware with example code, log files etc, so they may investigate.

I also sent you a Skype message, contact me if you need help.

Paul.

ClickAlgo
31 Jan 2016, 17:31

One other thing is that I would add clean up code to disconnect from the database, kill any session objects etc, even though the .NET framework uses a garbage collector to do all this, it is still good housekeeping to make you clean up after the party.

ChasBrownTH
03 Feb 2016, 07:32

RE:

Thanks Paul, as usual you are very helpful. MANY THANKS  ;)

At first I could not get the timer to close, but when I added it initialisation as a private variable my Robot built correctly, plus it closed the Timer at last! Also I have now added a mySQL connection.Close(). BUT if that had been there all along then I would not have known about the timer still being alive. Serendipity perhaps ?

       #region private fields

         // setup extra timer to invoke tasks each minute - checks environment etc
        private System.Timers.Timer OneMinuteTimer = new System.Timers.Timer();

         #endregion private fields

         OnStart()

         {

             // configure private 1 minute timer
            OneMinuteTimer.Elapsed += new ElapsedEventHandler(OnMinuteTimer);
            OneMinuteTimer.Interval = 60000;
            OneMinuteTimer.Enabled = true;

          }

         public void OnMinuteTimer(object sender, ElapsedEventArgs ev)

        {
            if (!IsBacktesting)
            {
                // WriteIsAliveToSQL();
                SQLInsertIsAliveInfo();)

            }

        }

          OnStop()

          {

                  OneMinuteTimer.Enabled = false;

                   connection.Close();  // close mySQL connection

          }

Paul_Hayes said:

declare this as a private variable in the class

     OneMinuteTimer.Enabled = false;

Now you can use the private variable in the scope of the class to stop it, even the OnStop event class of cAlgo, you are correct this will stop the timer and it is not threaded. There could be a possibility that when you stop the robot the instance is still in memory. IT WAS! The robot instance should really unload from memory when you stop the instance in cAlgo. IT DOESN'T!

One other thing is that I would add clean up code to disconnect from the database, kill any session objects etc, even though the .NET framework uses a garbage collector to do all this, it is still good housekeeping to make you clean up after the party.