CPU consumption is not at 100%
CPU consumption is not at 100%
16 May 2017, 14:17
Hello,
now i have bought a new PC for optimization, but it does not consume 100% CPU.
The performance slider is at 100% in cAlgo and it also all cores are used which is good and fine so far.
But overall it only uses about 25% CPU.
When i run CPU-Z for example it consumes 100% when benchmarking.
Has anyone an idea?
Thanks in advance.
Best Regards,
andi21
Replies
Spotware
16 May 2017, 14:37
Hi andi21,
Are you referring to cAlgo and back testing? If this is the case, if you want your code to utilize all the cores of your CPU then you will need to use parallel programming inside your cBot. . cAlgo cannot parallelize the backtesting process since it is sequencial by nature.
@Spotware
andi21
16 May 2017, 14:43
Hi Spotware,
i am refering to optimization (not backtesting - i know that in backtesting it only runs one bot instance and hence one core/main thread), so multiple bots run in parallel.
Like i said all cores are used, so that is fine. But overall it does not consume 100%, but only around 25%.
Hope you can help me with this.
Thanks.
@andi21
Spotware
16 May 2017, 16:56
Hi andi21,
Optimization should utilize all your CPU processors. Can you send us a sample cBot to see if we can reproduce your issue and identify the reason? Do you do something else in your cBot that could cause a performance bottleneck like reading or writing in a file?
@Spotware
andi21
16 May 2017, 17:20
Hi Spotware,
I have just tried one other SampleBot here from Bots-Section - it is the same problem (cpu consumption only at about 25%, but all cores are used correctly). So we can be sure that it does not depend on the code of my bot. It is something out of my scope.
Do you possibly have another idea?
@andi21
andi21
17 May 2017, 19:45
RE:
andi21 said:
Please, Spotware - have you got any advice for this problem for me?
Is it possible that the work is correctly separated to be processed on each core (that works good), but the actual workload so the passes / bots which run in parallel in optimization is limited to 4 bots / workloads etc.?
Why do i have this idea?
Because on my other pc which has 4 cores it works over all cores and with 100%.
But on my new pc it works also over all 16 cores BUT only with 25% which would be 4 cores / bots / workloads.
Could that be the problem?
Has anyone else encountered this problem on a pc with 8 or 16 or more cores?
@andi21
andi21
17 May 2017, 22:54
RE:
kricka said:
It's a very interesting question sent to the Spotware Team. The program utilizes 4 core 100% but only 25% of a 16 core CPU.
I have also made a very simple quick & dirty (could be much better i know, but it is q&d) sample console app to prove the situation:
using System;
using System.Threading.Tasks;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine(Environment.ProcessorCount);
if (true) // set to false for single core use
{
int zzz = Environment.ProcessorCount;
zzz = 4; // Comment this out to use all processors
Action[] z = new Action[zzz];
for (int xx = 0; xx < zzz; xx++)
{
z[xx] = Set;
}
Parallel.Invoke(z);
}
else
{
Set();
}
Console.ReadKey();
}
private static void Set()
{
long x = 5;
for (long i = 0; i < 12147483647; i++)
{
if (x == 4)
{
throw new Exception();
}
else
{
x++;
}
}
}
}
}
The program proved me (like i said in my first post, that it is neither a problem of my bot code, because i also tested another simple sample bot, nor it is a problem of my new pc, because bench of cpu-z also uses 100%), that 100% are possible without any problem:
-> if i use 1 core (if (false)) -> 7% are used (100 % / 16 cores = about 7%)
-> if i use processorcount (comment out the zzz = 4;) -> 100% are used
-> if i use FIX 4 cores (using zzz = 4): 25% are used ((100% / 16 cores) * 4 cores = about 25% -> like in calgo
so everything should be fine, but probably there is a problem in limiting the actual workload (see my post /forum/calgo-support/11535#7).
So all 16 cores are used, but the workload is only limited to 4 so the work of 4 is divided on 16 cores instead of a workload of 16 that is divided on 16 cores.
@andi21
andi21
18 May 2017, 16:00
RE:
Spotware said:
Hi andi21,
Can you send us a cBot that we could run and reproduce the problem?
Hi Spotware,
as an example i have used this bot: /algos/cbots/show/3
If you have further questions, please don't hesitate to contact me.
Thanks in advance.
Best Regards,
andi21
@andi21
Spotware
18 May 2017, 16:26
Hi andi21,
Thank you. Can you also tell us how your computer behaves when optimizing the following cBot?
using System;
using System.Linq;
using cAlgo.API;
using cAlgo.API.Indicators;
using cAlgo.API.Internals;
using cAlgo.Indicators;
namespace cAlgo
{
[Robot(TimeZone = TimeZones.UTC, AccessRights = AccessRights.None)]
public class CPUIntensiveBot : Robot
{
[Parameter(DefaultValue = 0.0)]
public double Parameter { get; set; }
protected override void OnStart()
{
// Put your initialization logic here
}
protected override void OnTick()
{
for (int i = 0; i < 1000000; i++)
{
for (int z = 0; z < 1000000; z++)
{
}
}
}
protected override void OnStop()
{
// Put your deinitialization logic here
}
}
}
@Spotware
andi21
18 May 2017, 16:59
Hi Spotware,
thank you.
I don't know how that can be, but to be sure i compiled and tried another run with your CPUIntensiveBot-Sample. And it is using all cores correctly like before and also it is using 100% now.
But that means the following i think:
- if cBot X has intensive code in OnTick, then 100% can be used (like in the CPUIntensiveBot-Sample with the two nested fors)
- if cBot X has "normal" code (for the cpu), then only 25% are used - but in this case the OnTick-Method from the bot has to be called in a more cpu-Intensive way from cAlgo-Internals, so that the cpu can use 100%, so Trades can be sure their optimization runs in full speed (all cores - which works great through the slider - and also at 100% - which does not work 100% i think).
Seeing forward for your response and thanks.
@andi21
andi21
18 May 2017, 17:06
RE:
andi21 said:
Hi Spotware,
thank you.
I don't know how that can be, but to be sure i compiled and tried another run with your CPUIntensiveBot-Sample. And it is using all cores correctly like before and also it is using 100% now.
But that means the following i think:
- if cBot X has intensive code in OnTick, then 100% can be used (like in the CPUIntensiveBot-Sample with the two nested fors)
- if cBot X has "normal" code (for the cpu), then only 25% are used - but in this case the OnTick-Method from the bot has to be called in a more cpu-Intensive way from cAlgo-Internals, so that the cpu can use 100%, so Trades can be sure their optimization runs in full speed (all cores - which works great through the slider - and also at 100% - which does not work 100% i think).
Seeing forward for your response and thanks.
And furthermore:
- if i use only a fix parameter for the double Parameter for example 0, then only 7 % is used, but that is fine, because only one bot so one core will be instantiated by cAlgo-Optimizer
- if i use a range of for example -0.5 till 1.5 then there are 20 Possiblities, so enough that 16 bot instance can be created to use all cores
@andi21
Spotware
19 May 2017, 15:09
Hi andi21,
If our sample has managed to utilize the CPU 100%, it means that there is no problem with cAlgo or your CPU. The performance issue could be expained by the fact that parallel programming is not as simple as it sounds.100% processor utilization can occur only for algogrithms that are fully parallelizable and utilize only the CPU, like the one we sent. A possible cause for the issue could be that there is some other bottleneck on your computer. For exampe, running your martingale algorithm on an eight core CPU we averaged an 80% CPU utilization.
@Spotware
andi21
19 May 2017, 15:21
Hi Spotware,
100% processor utilization can occur only for algogrithms that are fully parallelizable
Yes - and i think that should be the case if i set the performance slider to 100% in optimization and so there are 16 bot instances / runs in parallel running, isn't it?
@andi21
andi21
19 May 2017, 15:26
RE:
andi21 said:
Hi Spotware,
Yes - and if i set the performance slider in optimization to 100% it uses all cores and runs 16 bots / runs in parallel, shouldn't it be possible / use the full about 6,25% of each bot on each core?
Sorry for the double-post - a 404 error occured after sending the first post and i created a second one, but the first one was also created although an error occured.
@andi21
... Deleted by UFO ...