Optimization so slow it actually will not finish

Created at 04 Jun 2023, 00:30
How’s your experience with the cTrader Platform?
Your feedback is crucial to cTrader's development. Please take a few seconds to share your opinion and help us improve your trading experience. Thanks!
SH

Shares4us

Joined 01.04.2020

Optimization so slow it actually will not finish
04 Jun 2023, 00:30


If you write a tiny scalper and want to optimize it over a period of 6 or more months. ctrader optimization is soooo slooooowwww it actually FAILS!

What happens: 

scalps can be 5000 to 15000 orders per month per symbol.

ctrader for some reasons keeps its logs, events and history in memory.

thus your mem fills up and the system starts to swap => Enormous overhead and exceptional slow performance.

Is there a solution to this problem?

Can I switch off the events / log storage in optimizing or rerout it to disk or skip all those double/triple messages being generated?

Kill double messages example:

(EVENTS)0-1 Create Limit Order03/01/2023 02:30:00.591€ 1k0.01 LotsSell1.06760----10 000.00
(LOG)03/01/2023 02:30:00.591 | Placing Limit Order to Sell 1000 EURUSD (Price: 1.0676)
(LOG)03/01/2023 02:30:00.591 | → Placing Limit Order to Sell 1000 EURUSD (Price: 1.0676) SUCCEEDED, PendingOrder OID1

could become:
(EVENTS)0-1 Create Limit Order03/01/2023 02:30:00.591€ 1k0.01 LotsSell1.06760----10 000.00
or 
(LOG)03/01/2023 02:30:00.591 Placing Limit Order to Sell 1000 EURUSD (Price: 1.0676) SUCCEEDED, PendingOrder OID1

or when modifying:

(EVENTS) 5-1 Position Modified (T/P) 03/01/2023 03:15:22.773 € 1k 0.01 Lots Sell	1.06710	1.06410	-0.00 0-9 999.94
(LOG)03/01/2023 03:54:34.471 | Modifying position PID1 (SL: null, TP: 1.06336)
(LOG)03/01/2023 03:54:34.471 | → Modifying position PID1 (SL: null, TP: 1.06336) SUCCEEDED, Position PID1

could become:
 (EVENTS) 5-1 Position Modified (T/P) 03/01/2023 03:15:22.773 € 1k 0.01 Lots Sell	1.06710	1.06410	-0.00 0-9 999.94
or
(LOG)03/01/2023 03:54:34.471 Modifying position PID1 (SL: null, TP: 1.06336) SUCCEEDED, Position

Another solution could be:
If memory reaches a certain level start dump the stuff to disk ( we all have SSD nowadays). It is much more important to finish optimizing than being fast in checking out the results.

Yet another solution could be
Log, but when it went OK clear it from the log. ( if all goes well you will see the data in the orders/positions/history later on.

or combine all three of them ;-)

Best rgds

PS build  Memory management into optimization please?!?! and implement Start where you left of in optimizing!


@Shares4us