Tip for significant bot performance improvement
Tip for significant bot performance improvement
05 Sep 2019, 07:09
Hey everyone:
Just wanted to share a performance tip for those who program up their own bots that are running on a VPS with limited memory. With all the issues surrounding the OnBar recently, I decided to try replacing calls to Indicators in my bot code with the actual Indicator code itself.
This seems to be working extremely well if:
1) your bot doesn't need to graph anything
2) your bot doesn't need more than a few previous bars for data
Example:
One of my bots uses an OSMA indicator, of which I only ever need the 'current' value, and two previous values.
So what I've done is:
1) created an array with 3 slots
private double[] _osmaResultLongPeriod = new double[] { 0, 0, 0 };
2) created a method to initialize the values
//This is called via:
// InitializeOSMALongPeriod(_marketSeriesM5.Open.Count - 1);
//to prepopulate the previous 3 values
private void InitializeOSMASLongPeriod(int timeFrameIndex)
{
for (int x = 2; x >= 0; x--)
{
_osmaResultLongPeriod[x] = _osmaLongPeriod.Signal[timeFrameIndex - x] - _osmaLongPeriod.Histogram[timeFrameIndex - x];
}
}
3) created a method to update the values with each "bar" and "tick" which do the following:
//OnBar _osmaResultLongPeriod[2] = _osmaResultLongPeriod[1]; _osmaResultLongPeriod[1] = _osmaResultLongPeriod[0]; _osmaLongPeriodLastValue = _osmaResultLongPeriod[1]; //OnTick _osmaResultLongPeriod[0] = _osmaLongPeriod.Signal[currentIndexM5] - _osmaLongPeriod.Histogram[currentIndexM5]; _osmaLongPeriodCurrentValue = _osmaResultLongPeriod[0];
This removes all the IndicatorDataSeries objects, significantly saving on memory usage!
Running 20-30 instances of bots at once, following this technique has reduced memory usage over the course of a few hours by 500MB. Plus, straight up arrays appear to be significantly fast than accessing IndicatorDataSeries objects.
The added bonus is the single-CPU instance on the VPS is running more in the 60-80% range instead of 95%+, thus is more responsive.
So if anyone else has time to kill, I encourage others to try doing the same to remove the resource-hungry IndicatorDataSeries objects, especially if your bot only needs data values, and not actually graph anything.
Replies
firemyst
05 Sep 2019, 11:27
RE:
alexsanramon said:
If I understand you correctly, what you are saying is: instead of using GetIndicator<>, you place the calculation in the cbot code?
Exactly. Any you get from https://ctrader.com/algos/indicators
that includes the code you can put in your bot almost directly (depends on how you have your bot coded of course).
Even some that come with cTrader you can eliminate the GetIndicator<> call because the calculations are simple.
For example, the Momentum Indicator. It's calculation is just:
Momentum[index] = MarketSeries.Close[index] - MarketSeries.Close[index - Momentum_Period];
which you can easily insert into your bot code by creating your own array with however many bars back your bot will need to look. For instance, if all your bot does it look at the current value, and previous bar value, then you just need two values at any one time at most. So:
double[] Momentum = new double[] { 0, 0 }; //initialize
where Momentum[0] is the latest value (replicates "indicator.Result.Last(0)") and Momentum[1] is from 1 bar ago (replicates "indicator.Result.Last(1)").
Your single Momentum array will take up significantly less memory, never expand because it won't store more than 2 values, and will ultra fast for the CPU to deal with as opposed to the Momentum IndicatorDataSeries object which has a bigger memory footprint, is always growing with each OnBar, and takes more processing time for the CPU.
:-)
@firemyst
alexsanramon
06 Sep 2019, 12:56
I see that you have discovered the flaw in ctrader resource management design. Thanks for the tip. Although it would only work if you know the source code of all the indicators you are going to use. The problem is most indicators by ctrader are closed.
@alexsanramon
firemyst
06 Sep 2019, 16:13
RE:
alexsanramon said:
I see that you have discovered the flaw in ctrader resource management design. Thanks for the tip. Although it would only work if you know the source code of all the indicators you are going to use. The problem is most indicators by ctrader are closed.
They are closed yes. But as I said, you can easily look up their formulas.
For example, I gave you the Momentum one above.
The Keltner Channel can be calculated with the defaults as follows:
Middle Line: 20-day exponential moving average
Upper Channel Line: 20-day EMA + (2 x ATR(10))
Lower Channel Line: 20-day EMA - (2 x ATR(10))
Bollinger Bands are similar:
Middle Band = 20-day simple moving average (SMA)
Upper Band = 20-day SMA + (20-day standard deviation of price x 2)
Lower Band = 20-day SMA – (20-day standard deviation of price x 2)
Bollinger Bandwidth:
(Upper Bollinger Band - Lower Bollinger Band) / Middle Bollinger Band
Simple Moving Average:
(Price1 + Price 2 + ... + Price N) / N
etc etc.
@firemyst
alexsanramon
05 Sep 2019, 10:23
If I understand you correctly, what you are saying is: instead of using GetIndicator<>, you place the calculation in the cbot code?
@alexsanramon