Historical data on M1

Created at 31 Mar 2023, 16:38
How’s your experience with the cTrader Platform?
Your feedback is crucial to cTrader's development. Please take a few seconds to share your opinion and help us improve your trading experience. Thanks!
SW

swapd0

Joined 07.10.2019

Historical data on M1
31 Mar 2023, 16:38


I'm using ProtoOAGetTrendbarsReq to get historical data on different time frames, but on M1 I only get data for today candles, if I send a query to fetch yesterday candles I get no response back.


@swapd0
Replies

swapd0
01 Apr 2023, 22:14

This is weird, I send request in M1, H1 and D1 but I don't get M1 data, also usually I get some messages without any data, just a lot of zeros. H1 and D1 works but they are a bit slow, I have to wait about 10 seconds or more for a reply.

 


@swapd0

swapd0
02 Apr 2023, 15:25

Today, Sunday 2/4/2023, H1 request doesn't work, I haven't touched the code but all my H1 or M1 request gets a response full of zeros, D1 works.


@swapd0

Spotware
03 Apr 2023, 08:26

Dear trader,

Do you still experience this problem? During the weekends maintenance takes place so such issues might occur.

Best regards,

cTrader Team


@Spotware

swapd0
03 Apr 2023, 10:36

Yes, still it doesn't work.

I have the same problem as this user 

 


@swapd0

kostya.bartchenkov
03 Apr 2023, 14:49

I get the same issue. No code changes were done at all but ProtoOAGetTrendbarsReq stopped returning results.

This is for demo account, I tried multiple proxies - AU and EU, tried multiple bar sizes and pairs with the same result. Other calls seem to work.


@kostya.bartchenkov

Spotware
03 Apr 2023, 16:34

Dear traders,

In order to investigate this issue further, please provide us with the following information

  • Application id (client id)
  • IP address at the time of request
  • Exact time/date of the request
  • The exact data of the request
  • The exact data of the response

Best regards,

cTrader Team


@Spotware

swapd0
03 Apr 2023, 17:32 ( Updated at: 03 Apr 2023, 17:55 )

I'm connected to live-eu.ctraderapi.com, this is the request using ProtoOAGetTrendbarsReqm, right now I'm not using the set_count paramerter, but I get the same result if I use it.

I think that my IP is 92.57.56.65

clientId: 5666_hFOvRarXsCjgiR0DBMVVVeGbSWpZRcO5w5q1dVLCBcobxkzBnE

accountId: 16177258

symbol: AUDCAD(18)

period: 1 (1 minute)

from: 27/3/2023 0:00:00 (1679875200000)

end: 3/4/2023 16:25:07 (1680539107670)

sended at: 3/4/2023 16:24:59

 

This is the Response, a few chunck of zeros data.

unknow data (16383 bytes): 

memory dump: 0 0 0 0 0 0 0 0

unknow data (400e bytes): 

memory dump: 0 0 0 0 0 0 0 0

unknow data (4002 bytes): 

memory dump: 0 0 0 0 0 0 0 0

unknow data (3ffd bytes): 

memory dump: 0 0 0 0 0 0 0 0

unknow data (4 bytes): 

memory dump: 0

 


@swapd0

PanagiotisChar
04 Apr 2023, 08:53

Hi there,

In case Pepperstone is your broker, they had some issues yesterday.

Aieden Technologies

Need help? Join us on Telegram

Need premium support? Trade with us


@PanagiotisChar

swapd0
04 Apr 2023, 10:20

My broker is ICMarkets.


@swapd0

kostya.bartchenkov
04 Apr 2023, 16:02

Thank you. Mine was with Pepperstone. Saying that everything works fine now again


@kostya.bartchenkov

swapd0
06 Apr 2023, 19:03

Lucky you, this is still not working for M1, but H1 it works again.


@swapd0

swapd0
08 Apr 2023, 12:02

No one is using OpenAPI and ICMarkets to access historical data? This is a bit frustrating :(


@swapd0

kostya.bartchenkov
09 Apr 2023, 15:44

RE:

swapd0 said:

No one is using OpenAPI and ICMarkets to access historical data? This is a bit frustrating :(

I also have an icMarkets account . I may try it out tomorrow if I have some time. What PAIR are you trying?


@kostya.bartchenkov

swapd0
09 Apr 2023, 16:21 ( Updated at: 09 Apr 2023, 21:43 )

RE: RE:

kostya.bartchenkov said:

swapd0 said:

No one is using OpenAPI and ICMarkets to access historical data? This is a bit frustrating :(

I also have an icMarkets account . I may try it out tomorrow if I have some time. What PAIR are you trying?

Any forex pair, but usually I choose AUDCAD because it's the first one in the list.

 

Right now I'm trying to use tick data to generate the 1min candles, I use set_clientmsgid to set the pair name but when I read the response I don't get any clientmsgid data. If I inspect the memory the message contains my clientmsdid but it didnt't read it... weird.

 

Edited:

Ok, when I create a ProtoMessage from the sockets buffer it fails (return false) on some messages. I downloaded the .proto messages about one month ago, and the lastest change was four months ago.

This function works when I receive Application and account messages, symbol list, spot prices, but fails when I receive ProtoOAGetTrendbarsRes although on D1 request I can read the candles.

 

ProtoMessage createMessage(void *ptr, size_t size)
{
    ProtoMessage msg;

    bool ok = msg.ParseFromArray(ptr, (int)size);

    tools::Log().check(!ok).error("Parse from array failed: " + std::to_string(msg.payloadtype()));

    return msg;
}

 


@swapd0

swapd0
10 Apr 2023, 22:00

This is the code to read a message.

First I read an int32_t to know the message length, and then I create the message using ParseFromArray or ParseFromPartialArray, I've seen that this code fails because the message length for trend bars is wrong, I get a size of 21,508 bytes but if I inspect the memory there's about 16,391 bytes, after that is full of zeros.

 

	mConnection->receive(mBuffer.data(), mBuffer.size(),
	[&](std::error_code ec, std::size_t size)
	{
		if ( !ec )
		{
			int32_t length;
			memcpy(&length, mBuffer.data(), sizeof(int32_t));
			// big endian to little endian
			length = ntohl(length);

			if ( length > mBuffer.size() )
				tools::Log().error("Message too big: " + std::to_string(length));

			if ( length > 0 )
			{
				ProtoMessage pmsg(OpenMessageFactory::message(mBuffer.data() + sizeof(int32_t), length));
				int type = pmsg.payloadtype();
				auto it = mObservers.find(type);
				if ( it != mObservers.end() )
					it->second.fn(pmsg, it->second.context);
				else
					tools::Log().error("Message not handled: " + std::to_string(type) + "with size: " + std::to_string(length));
			}
			receiveMessages();
		}
		else
		{
			tools::Log().error("Receive data: " + ec.message() + " size: " + std::to_string(size));
		}
	});

 

 


@swapd0

swapd0
10 Apr 2023, 22:35

If I make a crap-hack to find the end of the message, and send the corrected length, it also fails. IMHO this means that ProtoOAGetTrendbarsRes messages are just wrong.

 


@swapd0

PanagiotisChar
11 Apr 2023, 07:51

Hi,

Are you able to reproduce the problem with any of the Open API samples? If yes, it would be easier for Spotware to investigate as there would be some common reference code to discuss upon.

Aieden Technologies

Need help? Join us on Telegram

Need premium support? Trade with us

 


@PanagiotisChar

kostya.bartchenkov
11 Apr 2023, 08:08

Just tried my ICMarkets account - works fine for me (used GBPUSD M5)


@kostya.bartchenkov

PanagiotisChar
11 Apr 2023, 08:16

It's it 99% something with the implementation. If there was a general issue with the API, there would be many complaints.

Aieden Technologies

Need help? Join us on Telegram

Need premium support? Trade with us


@PanagiotisChar

swapd0
11 Apr 2023, 10:06

RE:

kostya.bartchenkov said:

Just tried my ICMarkets account - works fine for me (used GBPUSD M5)

Yes, I also gets M5 candles, but it fails with M1.


@swapd0

swapd0
11 Apr 2023, 11:29

Ok, I've a stupid bug with fragmented messages.


@swapd0