cAlgo Crashes with Async and Await Methods

PanagiotisCharalampous
08 Nov 2017, 16:11

Hi myinvestmentsfx,

cAlgo does not crash, you just get a build error. You will get the same message if you open your indicator in Visual Studio. 

Best Regards,

Panagiotis

myinvestmentsfx
08 Nov 2017, 18:31

RE:

Panagiotis Charalampous said:

Hi myinvestmentsfx,

cAlgo does not crash, you just get a build error. You will get the same message if you open your indicator in Visual Studio. 

Best Regards,

Panagiotis

Hi Panagiotis,

Thank you for your promt response, I do however disagree slightly with you as this build perfectly within Visual Studio, but I need .Net Framework to be set to 4.6.1 within the project for the build to be succesful.  cAlgo, I believe uses .Net Framework 4.0? As soon as I add an instance to the indicator, cAlgo crashes.

Kind Regards,

myinvestmentsfx

ClickAlgo
08 Nov 2017, 18:38 ( Updated at: 21 Dec 2023, 09:20 )

Hi,

Your code gives this error when you attempt to build it in cAlgo

Error CS4033: The 'await' operator can only be used within an async method. Consider marking this method with the 'async' modifier and changing its return type to 'Task'.

Build errors in Visual Studio

The link below will tell you how to fix it.

https://stackoverflow.com/questions/11836325/await-operator-can-only-be-used-within-an-async-method

Paul Hayes
Sales & Marketing
Email: contact@clickalgo.com
Phone: (44) 203 289 6573
Website: https://clickalgo.com

myinvestmentsfx
08 Nov 2017, 21:16 ( Updated at: 21 Dec 2023, 09:20 )

RE:

Paul_Hayes said:

Hi,

Your code gives this error when you attempt to build it in cAlgo

Error CS4033: The 'await' operator can only be used within an async method. Consider marking this method with the 'async' modifier and changing its return type to 'Task'.

Build errors in Visual Studio

The link below will tell you how to fix it.

https://stackoverflow.com/questions/11836325/await-operator-can-only-be-used-within-an-async-method

 

Paul Hayes
Sales & Marketing
Email: contact@clickalgo.com
Phone: (44) 203 289 6573
Website: https://clickalgo.com

Hi Paul,

Thank you for this, it's much appriciated. However it still seems that the problem is related to the .Net Framework rather than the actual code.  If you change project to use .Net Framework 4.6.2, then it's builds without any errors in visual studio.  If I then add an instance to the indicator in cAlgo, cAglo crashes.  See screenshot of sucessful build.

ClickAlgo
08 Nov 2017, 21:37

ok, that makes sense. 

I noticed that you have no exception-handling in the code, one mistake and it could make the platform crash, especially as you are using threaded tasks.

You need to add try's and catches to each block of code as well as debug the code from the start so that when you add an instance it starts debugging each line of code if you do this you will find your error.

myinvestmentsfx
08 Nov 2017, 21:50

RE:

Paul_Hayes said:

ok, that makes sense. 

I noticed that you have no exception-handling in the code, one mistake and it could make the platform crash, especially as you are using threaded tasks.

You need to add try's and catches to each block of code as well as debug the code from the start so that when you add an instance it starts debugging each line of code if you do this you will find your error.

Thank you Paul, i will do that and see what I can find.  Appriciate the guidance.

Kind Regards,

PanagiotisCharalampous
09 Nov 2017, 09:44

Hi guys,

Indeed cAlgo works with .Net Framework 4.0 for compatibility reasons. We will upgrade the framework in a later version.

Best Regards,

Panagiotis

myinvestmentsfx
09 Nov 2017, 13:01 ( Updated at: 21 Dec 2023, 09:20 )

RE:

Panagiotis Charalampous said:

Hi guys,

Indeed cAlgo works with .Net Framework 4.0 for compatibility reasons. We will upgrade the framework in a later version.

Best Regards,

Panagiotis

Hi Panagiotis,

Thank you for the update,

Is multi threading still as problem for the cAglo API?  I've tried various multi-threading techniques and all seem to be having endless problems.

-Async and Await

-Parralel For Loops

-Kicking of various tasks

All of them seem to be a problem.  I suspect it being related to the thread below?

Kind Regards,

PanagiotisCharalampous
09 Nov 2017, 14:40

Hi myinvestmentsfx,

Multithreading in general was not and is not a problem for cAlgo. The discussion you posted inquires whether cAlgo application itself is multithreaded, more specifically if it runs the bots on separate threads than the main application, causing synchronization issues. The answer to this question is no, cAlgo is not running cBots in separate threads, except the Timer class, therefore there should not be any synchronization issues.

However, this does not mean that you cannot use multithreading techniques within your cBot. As long as your programming techniques are correct, then you should not experience any problems. If you want, you can post the issues you face one by one and we can discuss them.

Best Regards,

Panagiotis

myinvestmentsfx
09 Nov 2017, 15:54 ( Updated at: 21 Dec 2023, 09:20 )

Hi Panagiotis,

Ok, not a problem.  Please see the code used to test the various scenarios.

Attempt 1 and 2 can be run by commenting out one of the two under initialize.

  Test Results below,

using System;
using System.Linq;
using cAlgo.API;
using cAlgo.API.Indicators;
using cAlgo.API.Internals;
using cAlgo.Indicators;
using System.Collections.Generic;
using System.Threading.Tasks;
using System.Collections.Concurrent;

namespace cAlgo
{
    [Indicator(IsOverlay = false, TimeZone = TimeZones.UTC, AccessRights = AccessRights.None)]
    public class NewIndicator : Indicator
    {
        [Parameter(DefaultValue = 0.0)]
        public double Parameter { get; set; }

        [Output("Main")]
        public IndicatorDataSeries Result { get; set; }

        ///////////////////////////////////////////////////////////////////////////////////////////////////
        //                       Illustration Purposes
        //   Goal is to get the lows of multiple symbols in a specified timeframe at the same time
        //   
        ///////////////////////////////////////////////////////////////////////////////////////////////////


        protected override void Initialize()
        {
            ///////////////////////////////////////////////////////////////////////////////////////////////////
            //                       Attempt 1
            //   Calling MarketData.GetSeries & MarketSeries works fine together, 
            //   however 2 x MarketData.GetSeries calls Kicking off at the same time seems to be a problem.
            //   Uncomment below to run.
            ///////////////////////////////////////////////////////////////////////////////////////////////////

            // Attempt1_GetSymbolCodeLowAsyncWithTasks();

            ///////////////////////////////////////////////////////////////////////////////////////////////////
            //                       Attempt 2
            //   Parallel Foreach loop work fine with doing dummy calculation
            //   If same logic is applied but trying to get lows of multiple symbols, we run into a problem.
            //   Uncomment below to run.
            ///////////////////////////////////////////////////////////////////////////////////////////////////        

            Attempt_2_ParallelGetLows(AllSymbolCodes());
        }

        public override void Calculate(int index)
        {
            // Calculate value at specified index
            // Result[index] = ...
        }
        public void Attempt1_GetSymbolCodeLowAsyncWithTasks()
        {
            var t1 = Task.Factory.StartNew(() => EURUSD());
            var t2 = Task.Factory.StartNew(() => GBPUSD());
            var t3 = Task.Factory.StartNew(() => USDCHF());
            var t4 = Task.Factory.StartNew(() => USDJPY());
        }

        public double EURUSD()
        {
            Print("API Call 1 Kicking Off");
            double result = (MarketData.GetSeries(MarketData.GetSymbol("EURUSD"), TimeFrame.Daily).Low.Last(0));
            Print("API Call 1 EURUSD Low Result: ", result);
            return result;
        }
        public string GBPUSD()
        {
            Print("API Call 2 Kicking Off (Different API) ");
            string result = (MarketSeries.SymbolCode);
            Print("API Call 2 Get Symbol Code Result: ", result);
            return result;
        }
        public double USDCHF()
        {
            Print("Kicking off API Call 3");
            double result = (MarketData.GetSeries(MarketData.GetSymbol("USDCHF"), TimeFrame.Daily).Open.Last(0));
            Print("API Call 3 Get USDCHF Open Result: ", result);
            return result;
        }
        public double USDJPY()
        {
            Print("Kicking off API Call 4");
            double result = (MarketData.GetSeries(MarketData.GetSymbol("USDJPY"), TimeFrame.Daily).Close.Last(0));
            Print("API Call 4 Get USDJPY Close Result: ", result);
            return result;
        }


        public List<Symbol> AllSymbolCodes()
        {
            List<Symbol> allSymbolCodes = new List<Symbol> 
            {
                MarketData.GetSymbol("EURUSD"),
                MarketData.GetSymbol("GBPUSD"),
                MarketData.GetSymbol("USDCHF"),
                MarketData.GetSymbol("USDJPY"),
                MarketData.GetSymbol("USDCAD"),
                MarketData.GetSymbol("AUDUSD"),
                MarketData.GetSymbol("GBPJPY"),
                MarketData.GetSymbol("EURJPY"),
                MarketData.GetSymbol("NZDUSD"),
                MarketData.GetSymbol("AUDJPY")
            };
            return allSymbolCodes;
        }
        public ConcurrentBag<double> Attempt_2_ParallelGetLows(List<Symbol> allSymbolCodes)
        {
            var StartTime = Server.Time;
            ConcurrentBag<double> allSymbolCodesDaily = new ConcurrentBag<double>();

            Print("Starting Parallel Loop Testing ...");
            ConcurrentBag<double> TestParallel = new ConcurrentBag<double> 
            {
                1.1,
                2.2,
                3.3,
                4.4,
                5.5,
                6.6,
                7.7,
                8.8
            };
            ConcurrentBag<double> TestParallel2 = new ConcurrentBag<double>();

            //Testing Parallel For Loop with dummy data, seems to work fine.

            Parallel.ForEach(TestParallel, (double t) =>
            {
                double result = 5 * t;
                TestParallel2.Add(result);
            });
            Print("Ending Parralel Loop Test...");
            Print("Test Loop Results: ", string.Join(",", TestParallel2));

            //Get lows of multiple symbols using Parallel For Loop, doesn't seem to work.

            Print("Starting Parralel Loop Get Lows...");
            Parallel.ForEach(allSymbolCodes, (Symbol i) =>
            {
                double result = (MarketData.GetSeries(i, TimeFrame.Daily).Low.Last(0));
                allSymbolCodesDaily.Add(result);
            });
            Print("Ending Parralel Loop Get Lows...");
            Print(string.Join(",", allSymbolCodesDaily));

            return (allSymbolCodesDaily);
        }
    }
}

Result of Attempt 2 - The Get Lows Parallel Loop Never Finish

Attempt 1 - Call 3 & 4 never completes (Same type of API call as the first one)

myinvestmentsfx
09 Nov 2017, 22:57 ( Updated at: 21 Dec 2023, 09:20 )

After some try and catch, debugging etc.  I found this?  Any ideas what it means?

Kind Regards,

myinvestmentsfx

myinvestmentsfx
10 Nov 2017, 12:08

RE:

Panagiotis Charalampous said:

Hi myinvestmentsfx,

Multithreading in general was not and is not a problem for cAlgo. The discussion you posted inquires whether cAlgo application itself is multithreaded, more specifically if it runs the bots on separate threads than the main application, causing synchronization issues. The answer to this question is no, cAlgo is not running cBots in separate threads, except the Timer class, therefore there should not be any synchronization issues.

However, this does not mean that you cannot use multithreading techniques within your cBot. As long as your programming techniques are correct, then you should not experience any problems. If you want, you can post the issues you face one by one and we can discuss them.

Best Regards,

Panagiotis

Hi Panagiotis,

Any update on the information, testing and debug results I posted?

Kind Regards,

myinvestmentsfx

PanagiotisCharalampous
10 Nov 2017, 12:18

Hi myinvestmentsfx,

The product team is looking into it. I will update you as soon as I get their feedback.

Best Regards,

Panagiotis

myinvestmentsfx
10 Nov 2017, 13:07 ( Updated at: 21 Dec 2023, 09:20 )

RE:

Panagiotis Charalampous said:

Hi myinvestmentsfx,

The product team is looking into it. I will update you as soon as I get their feedback.

Best Regards,

Panagiotis

Thank you, much appriciated.  i've also asked another developer to look at the code and they tried using old school multithreading techniques.  cAlgo crashes and they get the error below.  Same error I go using Parallel loop.  See code below.

using System;
using System.Diagnostics;
using System.Linq;
using cAlgo.API;
using cAlgo.API.Indicators;
using cAlgo.API.Internals;
using cAlgo.Indicators;
using System.Collections.Generic;
using System.Threading.Tasks;
using System.Collections.Concurrent;
using System.Threading;

namespace cAlgo
{
    [Indicator(IsOverlay = false, TimeZone = TimeZones.UTC, AccessRights = AccessRights.None)]
    public class NewIndicator : Indicator
    {
        [Parameter(DefaultValue = 0.0)]
        public double Parameter { get; set; }

        [Output("Main")]
        public IndicatorDataSeries Result { get; set; }

        ///////////////////////////////////////////////////////////////////////////////////////////////////
        //                       Illustration Purposes
        //   Goal is to do get the lows of multiple symbols in a specified timeframe at the same time
        //   
        ///////////////////////////////////////////////////////////////////////////////////////////////////


        protected override void Initialize()
        {
            ///////////////////////////////////////////////////////////////////////////////////////////////////
            //                       Attempt 1
            //   Calling MarketData.GetSeries & MarketSeries works fine together, 
            //   however 2 x MarketData.GetSeries calls Kicking off at the same time seems to be a problem.
            //   Uncomment below to run.
            ///////////////////////////////////////////////////////////////////////////////////////////////////

            // Attempt1_GetSymbolCodeLowAsyncWithTasks();

            ///////////////////////////////////////////////////////////////////////////////////////////////////
            //                       Attempt 2
            //   Parallel Foreach loop work fine with doing normal calculation
            //   If same logic is applied but trying to get lows of multiple symbols, we run into a problem.
            //   Uncomment below to run.
            ///////////////////////////////////////////////////////////////////////////////////////////////////        

            // Attempt_2_ParallelGetLows(AllSymbolCodes());

            ///////////////////////////////////////////////////////////////////////////////////////////////////
            //                       Attempt 3
            //   Old school multithreading.
            //   Uncomment below to run.
            ///////////////////////////////////////////////////////////////////////////////////////////////////  

            test();

        }

        public override void Calculate(int index)
        {
            // Calculate value at specified index
            // Result[index] = ...
        }
        public void Attempt1_GetSymbolCodeLowAsyncWithTasks()
        {
            var t1 = Task.Factory.StartNew(() => EURUSD());
            var t2 = Task.Factory.StartNew(() => GBPUSD());
            var t3 = Task.Factory.StartNew(() => USDCHF());
            var t4 = Task.Factory.StartNew(() => USDJPY());
        }

        public double EURUSD()
        {
            Print("API Call 1 Kicking Off");
            double result = (MarketData.GetSeries(MarketData.GetSymbol("EURUSD"), TimeFrame.Daily).Low.Last(0));
            Print("API Call 1 EURUSD Low Result: ", result);
            return result;
        }

        public void EURUSD2()
        {
            Print("API Call 1 Kicking Off");
            double result = (MarketData.GetSeries(MarketData.GetSymbol("EURUSD"), TimeFrame.Daily).Low.Last(0));
            Print("API Call 1 EURUSD Low Result: ", result);
            // return result;
        }
        public string GBPUSD()
        {
            Print("API Call 2 Kicking Off (Different API) ");
            string result = (MarketSeries.SymbolCode);
            Print("API Call 2 Get Symbol Code Result: ", result);
            return result;
        }
        public double USDCHF()
        {
            Print("Kicking off API Call 3");
            double result = (MarketData.GetSeries(MarketData.GetSymbol("USDCHF"), TimeFrame.Daily).Open.Last(0));
            Print("API Call 3 Get USDCHF Open Result: ", result);
            return result;
        }

        public void USDCHF2()
        {
            Print("Kicking off API Call 3");
            double result = (MarketData.GetSeries(MarketData.GetSymbol("USDCHF"), TimeFrame.Daily).Open.Last(0));
            Print("API Call 3 Get USDCHF Open Result: ", result);
            // return result;
        }
        public double USDJPY()
        {
            Print("Kicking off API Call 4");
            double result = (MarketData.GetSeries(MarketData.GetSymbol("USDJPY"), TimeFrame.Daily).Close.Last(0));
            Print("API Call 4 Get USDJPY Close Result: ", result);
            return result;
        }


        public void USDJPY2()
        {
            Print("Kicking off API Call 4");
            double result = (MarketData.GetSeries(MarketData.GetSymbol("USDJPY"), TimeFrame.Daily).Close.Last(0));
            Print("API Call 4 Get USDJPY Close Result: ", result);
            // return result;
        }


        public ConcurrentBag<Symbol> AllSymbolCodes()
        {
            ConcurrentBag<Symbol> allSymbolCodes = new ConcurrentBag<Symbol> 
            {
                MarketData.GetSymbol("EURUSD"),
                MarketData.GetSymbol("GBPUSD"),
                MarketData.GetSymbol("USDCHF"),
                MarketData.GetSymbol("USDJPY"),
                MarketData.GetSymbol("USDCAD"),
                MarketData.GetSymbol("AUDUSD"),
                MarketData.GetSymbol("GBPJPY"),
                MarketData.GetSymbol("EURJPY"),
                MarketData.GetSymbol("NZDUSD"),
                MarketData.GetSymbol("AUDJPY")
            };
            return allSymbolCodes;
        }

        public void test()
        {
            try
            {
                ThreadStart threadactionJPN = new ThreadStart(USDJPY2);
                var threadJPN = new Thread(threadactionJPN);
                threadJPN.Start();

                ThreadStart threadactionEUR = new ThreadStart(EURUSD2);
                var threadEUR = new Thread(threadactionEUR);
                threadEUR.Start();

                ThreadStart threadactionCHF = new ThreadStart(USDCHF2);
                var threadCHF = new Thread(threadactionCHF);
                threadCHF.Start();
            } catch (Exception ex)
            {
                Debug.WriteLine("{0} Exception caught.", ex);
            }

        }
        public ConcurrentBag<double> Attempt_2_ParallelGetLows(ConcurrentBag<Symbol> allSymbolCodes)
        {
            var StartTime = Server.Time;
            ConcurrentBag<double> allSymbolCodesDaily = new ConcurrentBag<double>();

            //Get lows of multiple symbols using Parallel For Loop, doesn't seem to work.

            Print("Starting Parralel Loop Get Lows...");

            Parallel.ForEach(allSymbolCodes, (Symbol i) =>
            {
                try
                {
                    var x = (MarketData.GetSeries(i, TimeFrame.Daily));
                    try
                    {
                        double result = x.Low.Last(0);

                        try
                        {
                            allSymbolCodesDaily.Add(result);
                        } catch (Exception a)
                        {
                            Print("Problem occurred adding to list", a);
                        }
                    } catch (Exception b)
                    {
                        Print("Problem occured with getting low", b);
                    }
                } catch (Exception c)
                {
                    Debug.WriteLine("{0} Exception caught.", c);
                }

            });
            Print("Ending Parralel Loop Get Lows...");
            Print(string.Join(",", allSymbolCodesDaily));

            return (allSymbolCodesDaily);
        }
    }
}

Error Message Below.

.

myinvestmentsfx
11 Nov 2017, 18:31

RE:

Panagiotis Charalampous said:

Hi myinvestmentsfx,

The product team is looking into it. I will update you as soon as I get their feedback.

Best Regards,

Panagiotis

Hi Panagiotis,

I assume your product team doesn't work over the weekend?

Kind Regards,

myinvestmentsfx

myinvestmentsfx
13 Nov 2017, 10:46

RE: RE:

myinvestmentsfx said:

Panagiotis Charalampous said:

Hi myinvestmentsfx,

The product team is looking into it. I will update you as soon as I get their feedback.

Best Regards,

Panagiotis

Hi Panagiotis,

I assume your product team doesn't work over the weekend?

Kind Regards,

myinvestmentsfx

Hi Panagiotis,

Do you think we will receive some feedback from the product team today by any chance?

Kind Regards,

myinvestmentsfx

myinvestmentsfx
15 Nov 2017, 10:28

RE: RE: RE:

myinvestmentsfx said:

myinvestmentsfx said:

Panagiotis Charalampous said:

Hi myinvestmentsfx,

The product team is looking into it. I will update you as soon as I get their feedback.

Best Regards,

Panagiotis

Hi Panagiotis,

I assume your product team doesn't work over the weekend?

Kind Regards,

myinvestmentsfx

Hi Panagiotis,

Do you think we will receive some feedback from the product team today by any chance?

Kind Regards,

myinvestmentsfx

Hi Panagiotis,

Still nothing?

Kind Regards,

myinvestmentsfx

myinvestmentsfx
17 Nov 2017, 09:23

Hi Guys,

Anyone else in the community that can help with this?  Doesn't seem like Spotware has an answer for us?

Kind Regards,

myinvestmentsfx

PanagiotisCharalampous
20 Nov 2017, 10:55

Hi myinvestmentsfx,

I have some feedback from the product team. 

1. cAlgo APIs are not thread safe. All invocations of API methods should be done in main thread. If you are calling API method from another thread, you should use BeginInvokeOnMainThread() to wrap API call.
2. cAlgo is currently working on .Net version 4.0 to support early versions of Windows. We are planning to upgrade it soon. Since .Net version 4.0, code compilation at runtime has changed. Now it's compiled against latest version of .Net Framework installed on client PC. So you can use things like Async Await, but it's not supported by cAlgo.
   One of the issue with this is that context is not synchronized and you will return on another thread after awaiting method call.

Best Regards,

Panagiotis