Running a back test multiple times (Monte Carlo)
Running a back test multiple times (Monte Carlo)
08 Aug 2017, 06:49
I have the following simple cBot I would like to run a back test on 10,000 times, and aggregate the results so I can see a family of equity curves - does anyone have a suggestion on how to do this in an automated fashion in cAlgo ?
// -------------------------------------------------------------------------------------------------
//
// This code was a cAlgo API sample.
//
// Random entry, set s/l & t/p
using System;
using System.Linq;
using cAlgo.API;
using cAlgo.API.Indicators;
using cAlgo.API.Internals;
using cAlgo.Indicators;
namespace cAlgo
{
[Robot(TimeZone = TimeZones.UTC, AccessRights = AccessRights.None)]
public class Rnd_Ent_Set_SL_n_TP_20170808 : Robot
{
[Parameter("SL_PIPs", DefaultValue = -40)]
public int SL_PIPs { get; set; }
[Parameter("TP_PIPs", DefaultValue = 100)]
public double TP_PIPs { get; set; }
private const string label = "Rnd_Ent_Set_SL_n_TP_20170808";
Random random;
protected override void OnStart()
{
random = new Random();
}
protected override void OnTick()
{
var pos = Positions.FindAll(label, Symbol);
if (pos != null)
{
if (pos.Length == 0)
{
for (int i = 0; i < random.Next(25, 2500); i++)
{
random.Next(0, 2);
}
var tt = random.Next(0, 2) > 0 ? TradeType.Buy : TradeType.Sell;
var tr = ExecuteMarketOrder(tt, Symbol, Symbol.QuantityToVolume(0.01), label);
//Print("tr=", tr.Position.TradeType);
pos = Positions.FindAll(label, Symbol);
//Print("after trade pos.vol=", tr.Position.Volume);
}
else
{
var p = pos[0];
if (p.Pips > TP_PIPs || p.Pips < SL_PIPs)
ClosePosition(p);
}
}
}
}
}
Replies
aharonzbaida
16 Aug 2017, 22:50
RE:
solark said:
I'm pretty sure that's not currently possible. To do what you want to do I'd recommend adding a 'Seed' parameter, initialize the Random object using this parameter, then in the calgo "Optimization" tab, "optimize" over this parameter using "Grid Search" (Not genetic). This will get you a table of results you can then plot elsewhere (excel or whatever). Some notes:
1. Im not sure if the seed method will work properly, it depends on the random number generator. A random number generator with a big space of random numbers (for example Mersenne Twister) performs quite welll using a random seed (chance of 2 iterations running using the same sequence of 'random' numbers is very low). Another solution is to pre-generate a large file of random numbers and syncronize access to the file (the optimizer might run your algo in parallel). I think at 10000 times sequentials seeds should be fine but it's worth mentioning.
2. I can't remember how easy it is to export the table of results out of the optimizer. If it's a pain then just add some code `OnStop` to write the results to a file. If you're looking for equity curves you'll pretty much have to do this but it's not so bad (maybe a couple lines of code).
Hope that helps.
Solark, thank you for your reply - this is a clever idea. I am not sure how to acess the equity curve data in the optimizer from within the cBot OnStop method. The optimizer only keeps the best 20 runs, and doesn't give option of just dumping the raw data to disk.
@aharonzbaida
solark
16 Aug 2017, 23:43
Simplest is: (untested but looks right)
protected override void OnStop()
{
System.IO.File.WriteAllLines("filename", History.Select(x => String.Format("{0:o},{1}", x.ClosingTime, x.Balance)));
}
You'll want 'filename' to be an absolute path and maybe have the seed in the filename. This only makes datapoints at each trade, if you wanted to include unrealized you'd need to do something a bit different in the OnBar method and dump it OnStop.
Writing them to separate files creates a bit of a chore after the fact to aggregate the data. If python/r/or whatever is your tool of choice to ultimately plot the curves then this should be fine. If it's excel it might be easier to write the data as rows in the same csv (would require some locking to ensure proper writing) and then transpose the data once in excel (and possibly align dates).
@solark
solark
16 Aug 2017, 22:08
I'm pretty sure that's not currently possible. To do what you want to do I'd recommend adding a 'Seed' parameter, initialize the Random object using this parameter, then in the calgo "Optimization" tab, "optimize" over this parameter using "Grid Search" (Not genetic). This will get you a table of results you can then plot elsewhere (excel or whatever). Some notes:
1. Im not sure if the seed method will work properly, it depends on the random number generator. A random number generator with a big space of random numbers (for example Mersenne Twister) performs quite welll using a random seed (chance of 2 iterations running using the same sequence of 'random' numbers is very low). Another solution is to pre-generate a large file of random numbers and syncronize access to the file (the optimizer might run your algo in parallel). I think at 10000 times sequentials seeds should be fine but it's worth mentioning.
2. I can't remember how easy it is to export the table of results out of the optimizer. If it's a pain then just add some code `OnStop` to write the results to a file. If you're looking for equity curves you'll pretty much have to do this but it's not so bad (maybe a couple lines of code).
Hope that helps.
@solark