A big bug in tester - optimizer

alexander.n.fedorov
05 Mar 2019, 12:58 ( Updated at: 21 Dec 2023, 09:21 )

RE:

alexander.n.fedorov said:

Dear Panagiotis!

It looks like there is a big bug in tester-optimizer/

The way I see the optimization is done is this

1. You do optimization with a set of parameters

2. You  select certain result and push the "Apply" green button

3. when you do that all the timing, onTick, comissions and parameters are copied to backtester

4. You run backtest and , theoretically, and as it always was happening, must get the same results

in this case it did not work like that

see the images:

PanagiotisCharalampous
05 Mar 2019, 14:05

Hi Sasha,

It is probably related to this issue.

Best Regards,

Panagiotis

alexander.n.fedorov
05 Mar 2019, 14:09

RE:

Panagiotis Charalampous said:

Hi Sasha,

It is probably related to this issue.

Best Regards,

Panagiotis

So, it is known bug?

PanagiotisCharalampous
05 Mar 2019, 14:18

Hi Sasha,

The issue in the thread I posted is a bug and will be fixed. However I don't know if this is your case as well since I do not have the source code. But probably it is caused by the same issue.

Best Regards,

Panagiotis

alexander.n.fedorov
05 Mar 2019, 14:24

There is no prob to give you source code. It is just a simple Big Shadow strategy

I was trying to use optimizor, cause different pairs have differen behaviour

But, if you need code, just tell me, no prob

PanagiotisCharalampous
05 Mar 2019, 14:32

Hi Sasha,

This happens only when you are using data from another timeframe. If you do so then this is the reason.

Best Regards,

Panagiotis

alexander.n.fedorov
05 Mar 2019, 21:26

RE:

Panagiotis Charalampous said:

Hi Sasha,

This happens only when you are using data from another timeframe. If you do so then this is the reason.

Best Regards,

Panagiotis

TimeFrame is the same

PanagiotisCharalampous
06 Mar 2019, 10:08

Hi Sasha,

Can you please share the code then?

Best Regards,

Panagiotis

alexander.n.fedorov
06 Mar 2019, 11:53

RE:

Panagiotis Charalampous said:

Hi Sasha,

Can you please share the code then?

Best Regards,

Panagiotis

I sent it to  you

PanagiotisCharalampous
07 Mar 2019, 16:59

Hi Sasha,

I have checked your cBot and you are using data from another timeframe

       [Parameter("S & R TimeFrame", DefaultValue = "Daily")]
        public TimeFrame SRTimframe { get; set; }
.
.
.

     seriesSR = MarketData.GetSeries(SRTimframe);
.
.
.
.

Best Regards,

Panagiotis

alexander.n.fedorov
07 Mar 2019, 17:36

RE:

Panagiotis Charalampous said:

Hi Sasha,

I have checked your cBot and you are using data from another timeframe

       [Parameter("S & R TimeFrame", DefaultValue = "Daily")]
        public TimeFrame SRTimframe { get; set; }
.
.
.

     seriesSR = MarketData.GetSeries(SRTimframe);
.
.
.
.

Best Regards,

Panagiotis

O , that is tru. I thought you were saying that I do backtesting on different timeframe, then optimization

But that is still a bug

Best regards,

Alexander

 

 

alexander.n.fedorov
07 Mar 2019, 17:37

RE: RE:

alexander.n.fedorov said:

Panagiotis Charalampous said:

Hi Sasha,

I have checked your cBot and you are using data from another timeframe

       [Parameter("S & R TimeFrame", DefaultValue = "Daily")]
        public TimeFrame SRTimframe { get; set; }
.
.
.

     seriesSR = MarketData.GetSeries(SRTimframe);
.
.
.
.

Best Regards,

Panagiotis

O , that is tru. I thought you were saying that I do backtesting on different timeframe, then optimization

But that is still a bug

Best regards,

Alexander

P. S. And thank you very much for checking

 

 

 

 

PanagiotisCharalampous
07 Mar 2019, 17:55

Hi Sasha,

Yes it is a bug and we are aware about it. I just wanted to make sure it is the one we know. It happens on optimization when cBots get data form other timeframes than the one they execute on.

Best Regards,

Panagiotis

tradermatrix
02 Apr 2019, 12:20

yes it's a bug ..
I hulled all your bot piece by piece but finally I did not find any error ..
so i figured that the optimization flaw came up when the cBot gets data in the form of deadlines other than the one on which it runs.
I modified your code a little and I realized to the optimization that when my setting "Time Frame" is identical to the one on which it is executed there is no more problem ...
that's why it worked previously because I optimized on 5m ....
thanks to you I just noticed that some of my robots do not have the right settings ... because I trusted optimization.
hope that this problem will be settled soon
cordially

using System;
using System.Linq;
using cAlgo.API;
using cAlgo.API.Indicators;
using cAlgo.API.Internals;
using cAlgo.Indicators;

namespace cAlgo.Robots
{
    [Robot(TimeZone = TimeZones.RussianStandardTime, AccessRights = AccessRights.FullAccess)]
    public class VolatilityBreakout4 : Robot
    {
        [Parameter("Source")]
        public DataSeries Source { get; set; }

        [Parameter("Time Frame", DefaultValue = "Minute5")]
        public TimeFrame timeframe { get; set; }

        [Parameter("Risk", MaxValue = 10, MinValue = 0.5, DefaultValue = 1)]
        public double Risk { get; set; }

        [Parameter("Squeeze bars number", MaxValue = 180, MinValue = 5, Step = 1, DefaultValue = 20)]
        public int SqueezeBarsNumber { get; set; }

        [Parameter("Squeeze in pips", MaxValue = 300, MinValue = 1, Step = 1, DefaultValue = 20)]
        public int Squeeze { get; set; }

        [Parameter("TP multiple", MaxValue = 8, MinValue = 1, Step = 0.1, DefaultValue = 2)]
        public double TpMultiple { get; set; }

        [Parameter("Close on Profit %", Step = 0.1, DefaultValue = 4.0)]
        public double CloseOnProfit { get; set; }

        [Parameter("Close Trade Hour", DefaultValue = 23)]
        public int CloseTradesHour { get; set; }

        [Parameter("Close Trade Minute", DefaultValue = 30)]
        public int CloseTradesMinute { get; set; }

        [Parameter("Open Trade Hour", DefaultValue = 2)]
        public int OpenTradesHour { get; set; }

        private string instance;
        private MarketSeries series;
        private double range, high, low, volume, barsRange, SL, TP, minVolume, risk, longPrice;
        private double shortPrice, closeOnProfit;
        private int pipDigits, n;
        private DateTime squeezeFirstBarTime, squeezeLastBarTime;

        protected override void OnStart()
        {
            instance = ToString() + ", " + Symbol.Code + ", " + TimeFrame + ", " + Account.BrokerName + ", " + Account.Number;
            range = Squeeze * Symbol.PipSize;

            series = MarketData.GetSeries(timeframe);
            minVolume = Symbol.VolumeInUnitsMin;
            risk = Risk * 0.01;
            closeOnProfit = CloseOnProfit * 0.01;
            pipDigits = Symbol.Digits - (int)Math.Log10(1 / Symbol.PipSize);
            Positions.Opened += Positions_Opened;
            Positions.Closed += PositionsOnClosed;
            squeezeFirstBarTime = CandleOpenTime(2 + SqueezeBarsNumber);
            squeezeLastBarTime = CandleOpenTime(2);
        }

        protected override void OnBar()
        {
            var position = Positions.Find(instance);
            var spread = Symbol.Spread / Symbol.PipSize;
            if (position == null)
            {
                if (spread > 2)
                {
                    return;
                }
                if ((CandleOpenTime(0).Hour >= CloseTradesHour && CandleOpenTime(0).Minute >= CloseTradesMinute) || CandleOpenTime(0).Hour < OpenTradesHour)
                {
                    return;
                }
                squeezeFirstBarTime = CandleOpenTime(1 + SqueezeBarsNumber);
                squeezeLastBarTime = CandleOpenTime(1);
                high = CandleHigh(1);
                low = CandleLow(1);
                barsRange = CandleClose(1) > (1) ? MarketSeries.Close.Last(1) - MarketSeries.Open.Last(1) : MarketSeries.Open.Last(1) - MarketSeries.Close.Last(1);
                for (int i = 0; i < SqueezeBarsNumber; i++)
                {
                    high = CandleHigh(i + 1) > high ? CandleHigh(i + 1) : high;
                    low = CandleLow(i + 1) < low ? CandleLow(i + 1) : low;
                    barsRange = high - low;
                }
                //Chart.RemoveAllObjects();
                //Chart.DrawTrendLine("sHigh", squeezeFirstBarTime, high, squeezeLastBarTime, high, Color.Aqua, 1, LineStyle.Solid);
                //Chart.DrawTrendLine("sLow", squeezeFirstBarTime, low, squeezeLastBarTime, low, Color.Red, 1, LineStyle.Solid);
                //double barRangePips = Math.Round(barsRange / Symbol.PipSize, 1);
                //string text = "Squeeze = " + Squeeze + " , barRange = " + barRangePips;
                //Chart.DrawStaticText("Current range", text, VerticalAlignment.Bottom, HorizontalAlignment.Right, Color.Aquamarine);


                n = 0;
                foreach (var order in PendingOrders)
                {
                    if (order.Label == instance)
                    {
                        n = n + 1;
                    }
                }
                if (n == 0)
                {
                    if (high - low < range)
                    {
                        {
                            longPrice = high;
                            shortPrice = low;
                            SL = (high - low) / Symbol.PipSize + 1;
                            volume = Math.Floor(((Account.Equity * risk) / SL) / Symbol.PipValue / minVolume) * minVolume;
                            SL = Math.Round(SL, pipDigits, MidpointRounding.AwayFromZero);
                            TP = SL * TpMultiple;
                            PlaceStopOrder(TradeType.Buy, Symbol, volume, longPrice, instance, null, TP);
                            PlaceStopOrder(TradeType.Sell, Symbol, volume, shortPrice, instance, null, TP);
                            //Chart.DrawHorizontalLine(longPrice.ToString(), longPrice - SL * Symbol.PipSize, Color.Crimson, 1, LineStyle.DotsRare);
                            //Chart.DrawHorizontalLine(shortPrice.ToString(), shortPrice + SL * Symbol.PipSize, Color.Crimson, 1, LineStyle.DotsRare);
                        }
                    }
                }
            }
            if (position != null)
            {
                if (spread > 2)
                {
                    return;
                }
                if (position.TradeType == TradeType.Buy)
                {
                    if (CandleClose(1) < position.EntryPrice - SL * Symbol.PipSize)
                    {
                        ClosePosition(position);
                        //Chart.RemoveAllObjects();
                    }
                }
                if (position.TradeType == TradeType.Sell)
                {
                    if (CandleClose(1) > position.EntryPrice + SL * Symbol.PipSize)
                    {

                        ClosePosition(position);
                        //Chart.RemoveAllObjects();
                    }
                }
            }
        }
        protected override void OnTick()
        {
            if (Account.UnrealizedNetProfit > Account.Balance * closeOnProfit)
                foreach (var position in Positions)
                {
                    ClosePositionAsync(position);
                }
        }
        private void Positions_Opened(PositionOpenedEventArgs args)
        {
            var position = args.Position;
            if (position.Label == instance)
            {
                foreach (var order in PendingOrders)
                {
                    if (order.Label == instance)
                        CancelPendingOrder(order);
                }
                if (position.TradeType == TradeType.Buy)
                {
                    //Chart.RemoveObject(shortPrice.ToString());
                }
                if (position.TradeType == TradeType.Sell)
                {
                    //Chart.RemoveObject(longPrice.ToString());
                }
            }

        }
        private void PositionsOnClosed(PositionClosedEventArgs args)
        {
            var position = args.Position;

            if (args.Reason == PositionCloseReason.TakeProfit)
            {
                //
            }
        }
        #region Candles Processing
        #region Candle Type
        private bool CandleTypeBullish(int k)
        {
            if (series.Open.Last(k) > series.Close.Last(k))
            {
                return false;
            }
            return true;
        }
        #endregion

        #region Candle Range(int k)
        private double CandleRange(int k)
        {
            return series.High.Last(k) - series.Low.Last(k);
        }
        #endregion

        #region Candle body bottom
        private double CandleBodyBottom(int k)
        {
            return (series.Open.Last(k) < series.Close.Last(k)) ? series.Open.Last(k) : series.Close.Last(k);
        }
        #endregion

        #region Candle body Top
        private double CandleBodyTop(int k)
        {
            return (series.Open.Last(k) > series.Close.Last(k)) ? series.Open.Last(k) : series.Close.Last(k);
        }
        #endregion

        #region Candle high
        private double CandleHigh(int k)
        {
            return series.High.Last(k);
        }
        #endregion

        #region Candle low
        private double CandleLow(int k)
        {
            return series.Low.Last(k);
        }
        #endregion

        #region Candle body
        private double CandleBody(int k)
        {
            return Math.Abs((series.Open.Last(k) - series.Close.Last(k)));
        }
        #endregion

        #region Candle Close
        private double CandleClose(int k)
        {
            if (CandleTypeBullish(k))
            {
                return CandleBodyTop(k);
            }
            return CandleBodyBottom(k);
        }
        #endregion

        #region Candle Open
        private double CandleOpen(int k)
        {
            if (CandleTypeBullish(k))
            {
                return CandleBodyBottom(k);
            }
            return CandleBodyTop(k);
        }

        #endregion

        #region Candle Sinus
        private double CandleSinus(int k)
        {
            return Math.Abs((series.Open.Last(k) - series.Close.Last(k))) / CandleRange(k) * (series.Close.Last(k) > series.Open.Last(k) ? 1 : -1);
        }
        #endregion

        #region Trend Angle
        private double TrendAngle(int CandlesInTrend)
        {
            double absBody = 0;
            double bodiesRange = 0;
            for (int i = 0; i <= CandlesInTrend; i++)
            {
                if (CandleTypeBullish(i))
                {
                    bodiesRange = bodiesRange + CandleBody(i);
                }
                if (!CandleTypeBullish(i))
                {
                    bodiesRange = bodiesRange - CandleBody(i);
                }
                absBody = absBody + CandleBody(i);
            }
            var trendSinus = bodiesRange / absBody;
            return Math.Asin(trendSinus) * 180 / Math.PI;
        }
        #endregion

        #region Candle OpenTime
        private DateTime CandleOpenTime(int k)
        {
            return series.OpenTime.Last(k);
        }
        #endregion

        #region No k Candles on the left High
        private bool NoLeftCandlesHigh(int k)
        {
            for (int i = 1; i < k; i++)
            {
                if (CandleHigh(1) > CandleLow(1 + i) && CandleHigh(1) < CandleHigh(1 + i))
                {
                    return false;
                }
                if (CandleHigh(1) < CandleLow(1 + i) || CandleHigh(1) > CandleHigh(1 + i))
                {
                    continue;
                }
            }
            return true;
        }
        #endregion
        #region No k Candles on the left Low
        private bool NoLeftCandlesLow(int k)
        {
            for (int i = 1; i < k; i++)
            {
                if (CandleLow(1) < CandleHigh(1 + i) && CandleLow(1) > CandleLow(1 + i))
                {
                    return false;
                }
                if (CandleLow(1) < CandleHigh(1 + i) || CandleLow(1) > CandleLow(1 + i))
                {
                    continue;
                }
            }
            return true;
        }
        #endregion
        #endregion
    }
}

alexander.n.fedorov
03 Apr 2019, 13:49 ( Updated at: 21 Dec 2023, 09:21 )

Hi, TraDerMatTix

Be carefull with using different timeframe then the default is. You may face the same problem
My bot  is supposingly doing good results, and the optimization is nor serving a curve fitting, but rather makes aware of every pair and timeframe characteristik kind of "wavelength"

You have to run a portfolio of a 15-20 instruments.

Optimization should be not done before the date I put for a couple of reasons:

1. Brexit affected all the results in 2016 (checked it)

2. 2 years and a quater give  you rather representative stats on the wavelenth

What  i do first, I do multitimeframe optimization on 1 minute bars, then get the best timeframe based on pre-analysis, the live it overnight on tick in the best timeframe

With the risk structure I put, I normally accept a pair/timeframe if gives me a return of >=60% for 2.25.years. 

Then I take the optimized portfolio and consolidate the trades in Excel

This is the kind of curve that I receive

As you cand see with the original deposit of 5K, you should tripple your account in a matter of 4 month.

That should be statistically representative. So, now I am running a live test on Demo

Should find it usefull, you can try it . If you find some improvements, please, let me know

Alexander