OnException for an an error in Initialize of an Indicator appears to run after all the history Calculate calls

PanagiotisCharalampous
13 Oct 2024, 06:08

Hi there,

Can you share a code example so that we can understand the issue?

Best regards,

Panagiotis

martins
13 Oct 2024, 22:37

RE: OnException for an an error in Initialize of an Indicator appears to run after all the history Calculate calls

PanagiotisCharalampous said: 

Hi there,

Can you share a code example so that we can understand the issue?

Best regards,

Panagiotis

Try using the demonstator below: Clear the Trade Chart's Log after starting this with the defaults (which is no exceptions & display a green line .002 above the highs). Then click its Properties or right click the line, and change ‘Exception In Init?’ to Yes. The display will show Highs without the offset added (because the offset isn't copied from the parm until after the error point). Click ok to get rid of the parameter window so you can see the log. On my system the ‘OnException:’ messages don't appear until after all the history Calculates. Similarly, if I instead set ‘Exception In Calc’ to say 1140 (having seen there were 1150 history bars in the earlier log) then the OnException log for that doesn't appear until after all the history Calculates, ie around 1150 (visible on the screen because the default is to leave the indicator Result below Low, rather than above High, for Calculate calls that have exceptions (can make all of them fail by specifying -1 for ‘Exception In Calc’, or only all calls startinf at the 1140th by setting -1140).

Same happens on Backtesting chart, with the indicator added to an already complete backtest. 
The original place I noticed this was with an Indicator used by a Robot in Backtesting - in that case, the OnException happens after the default 200-bar history has been run through Calculate, i.e. when the (backtest) Server.Date as seen by the Robot is at the start date of the backtest.       

As an aside, why does the display line disappear if its IndicatorDataSeries is explicitly CreateDataSeries'ed? (the ‘call CreateDataSeries in Init?’ parm stuff). I was originally doing that in Initialize in another indicator in order to Print its .Count (even though the values would get overwritten later by Calculate) which - the use of name.Count in Initialize without a prior CreateDataSeries - causes an exception (that's what started all this!)

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using cAlgo.API;
using cAlgo.API.Collections;
using cAlgo.API.Indicators;
using cAlgo.API.Internals;

namespace cAlgo
{
    [Indicator(IsOverlay = true, AutoRescale = false, AccessRights = AccessRights.None)]
    public class mjsDemo : Indicator
    {
        [Parameter("Offset to add to High", DefaultValue = "0.002")]
        public double offsetParm { get; set; }
        [Parameter("Exception in Init?", DefaultValue = "false")]
        public bool doExceptionInInit { get; set; }
        [Parameter("Exception in Calc # (0=never 3=third etc or -n = all starting at n)", DefaultValue = "0")]
        public int doExceptionInCalc { get; set; }
        [Parameter("call CreateDataSeries in Init?", DefaultValue = "false")]
        public bool doCreateDataSeries { get; set; }
        [Output("Main")]
        public IndicatorDataSeries Result { get; set; }
        double offset = 0;
        string whereAreWe = "nowhere";
        protected override void Initialize()
        {
            Print("Initialize: started");
            whereAreWe = "Initialize";  // or System.Reflection.MethodBase.GetCurrentMethod().Name;
            Print("Initialize: whereAreWe=" + whereAreWe);
            if (doExceptionInInit) {
                Print("Initialize: throw exception next");
                throw new DivideByZeroException();
            }
            Print("Initialize: copy offsetParm to offset next");
            offset = offsetParm;
            Print("Initialize: offset=" + offset);
            if (doCreateDataSeries) {
                Print("Calculate: CreateDataSeries next");
                Result = CreateDataSeries();
                Print("Calculate: CreateDataSeries done");
            }
            whereAreWe = "nowhere";
            Print("Initialize: ending");
        }
        int countCalcs = 0;
        public override void Calculate(int index)
        {
            Print("Calculate: started index=" + index + " Bars[index].OpenTime=" + Bars[index].OpenTime + " offset=" + offset);
            if (whereAreWe != "nowhere") Print("Calculate: not nowhere so OnException hasn't run yet after an error in... whereAreWe=" + whereAreWe);
            whereAreWe = "Calculate" + index;   // or System.Reflection.MethodBase.GetCurrentMethod().Name + index;
            Print("Calculate: whereAreWe=" + whereAreWe);
            countCalcs++;
            Result[index] = Bars[index].Low - offset;   // will be overwritten if no exception
            Print("Calculate: temp Result[index]=" + Result[index]);
            if (countCalcs == doExceptionInCalc || (doExceptionInCalc < 0 && countCalcs > -doExceptionInCalc)) {
                Print("Calculate: throw exception next because doExceptionInCalc=" + doExceptionInCalc + " and countCalcs=" + countCalcs);
                throw new DivideByZeroException();
            }
            Result[index] = Bars[index].High + offset;
            Print("Calculate: offset Result[index]=" + Result[index]);
            whereAreWe = "nowhere";
            Print("Calculate: ending index=" + index);
        }
        protected override void OnException(Exception e)
        {
            Print("OnException: started exception=" + e);
            Print("OnException: last set whereAreWe=" + whereAreWe);
            Print("OnException: What can we do in here if history calcs are done already?");
            Print("OnException: Better to put try/catch around Init logic?");
            Print("OnException: copy offsetParm to offset next");
            offset = offsetParm;
            Print("Initialize: offset=" + offset);
            whereAreWe = "nowhere";
            Print("OnException: ending");
        }
    }
}

martins
16 Oct 2024, 21:43

RE: RE: OnException for an an error in Initialize of an Indicator appears to run after all the history Calculate calls

Did you see the same - it's as if, when Initialize runs, the 200 history calls to Calculate are already on whatever .NET scheduling queue c# uses, and any unhandled errors (i.e. without a Try/Catch) result in OnException calls aded to the end of the queue, whereas one might expect OnException to at least run in front of all pre-scheduled stuff, or ideally as an interrupt to the current task.

PanagiotisCharalampous
17 Oct 2024, 09:37

Hi there,

The OnException method was designed to inform developer that something is wrong in the algo. It is not supposed to be used in the execution flow of the algo. The order of execution of the OnException is not guaranteed and can be changed later. If you want to handle exceptions in predicted way you should use try catch {} blocks explicitly and do not rely on the OnException handler.

Best regards,

Panagiotis

martins
17 Oct 2024, 14:05 ( Updated at: 18 Oct 2024, 05:19 )

RE: OnException for an an error in Initialize of an Indicator appears to run after all the history Calculate calls

PanagiotisCharalampous said: 

Hi there,

The OnException method was designed to inform developer that something is wrong in the algo. It is not supposed to be used in the execution flow of the algo. The order of execution of the OnException is not guaranteed and can be changed later. If you want to handle exceptions in predicted way you should use try catch {} blocks explicitly and do not rely on the OnException handler.

Best regards,

Panagiotis

Fair enough, for Indicators possibly, although if a Robot relies on them they need the same fault tolerance. 
Does it work differently for Robots - reading this https://help.ctrader.com/ctrader-algo/fault-tolerance/ it presents OnException as if it can take the place of individual error handling: “you can also customise how your algos react to certain exceptions, or overwrite the default fault tolerance behaviors entirely. To do so, use the OnException() method. It acts as the dedicated handler for any unhandled exceptions.”

PanagiotisCharalampous
18 Oct 2024, 05:33

RE: RE: OnException for an an error in Initialize of an Indicator appears to run after all the history Calculate calls

martins said: 

PanagiotisCharalampous said: 

Hi there,

The OnException method was designed to inform developer that something is wrong in the algo. It is not supposed to be used in the execution flow of the algo. The order of execution of the OnException is not guaranteed and can be changed later. If you want to handle exceptions in predicted way you should use try catch {} blocks explicitly and do not rely on the OnException handler.

Best regards,

Panagiotis

Fair enough, for Indicators possibly, although if a Robot relies on them they need the same fault tolerance. 
Does it work differently for Robots - reading this https://help.ctrader.com/ctrader-algo/fault-tolerance/ it presents OnException as if it can take the place of individual error handling: “you can also customise how your algos react to certain exceptions, or overwrite the default fault tolerance behaviors entirely. To do so, use the OnException() method. It acts as the dedicated handler for any unhandled exceptions.”

Hi there,

Fault tolerance and correct exception handling is not the same thing. Fault tolerance means that the cBot/Indicator will continue running even if an unforeseen exception is thrown. It's there to help you from unpredicted exceptions. It's just a last resort handler in case an exception has not been explicitly handled by the programmer. Nevertheless this does not mean you do not need to handle properly exceptions you can foresee in your code. So do not rely on OnException() method for error handling.

Best regards,

Panagiotis