FIX RequestForPositions return only part from positions

Created at 30 Mar 2017, 22:28
How’s your experience with the cTrader Platform?
Your feedback is crucial to cTrader's development. Please take a few seconds to share your opinion and help us improve your trading experience. Thanks!
mindbreaker's avatar

mindbreaker

Joined 19.07.2014

FIX RequestForPositions return only part from positions
30 Mar 2017, 22:28


Hi Spotware,

when i click on Request for positions server send to me not all positions. Why?

First click send me positions from one symbol

Second click from second symbol

Something is wrong.

Code with Queue (or maybe i have got mistake but where)

using FIX_API_Library;
using System;
using System.Net.Security;
using System.Net.Sockets;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using System.Threading;
using System.Windows.Forms;
using System.Collections;
using System.Threading;

namespace FIX_API_Sample
{
    public partial class frmFIXAPISample : Form
    {
        private int _pricePort = 5211;
        private int _tradePort = 5212;
        /*
        private string _host = "h28.p.ctrader.com";
        private string _username = "3006156";
        private string _password = "sp0tw@re";
        private string _senderCompID = "sales.3006156";
        private string _senderSubID = "3006156";
        */

        private string _host = "h23.p.ctrader.com";
        private string _username = "9998871";
        private string _password = "5065";
        private string _senderCompID = "fxpro.9998871";
        private string _senderSubID = "9998871";
        
        private string _targetCompID = "CSERVER";

        // Add queue with messages
        public static Queue xQueuePrice = new Queue();
        public static Queue xQueueTrade = new Queue();

        private int _messageSequenceNumber = 1;
        private int _testRequestID = 1;
        private TcpClient _priceClient;
        private SslStream _priceStreamSSL;
        private TcpClient _tradeClient;
        private SslStream _tradeStreamSSL;
        private MessageConstructor _messageConstructor;

        public frmFIXAPISample()
        {
            InitializeComponent();
            try{

                /*
                Thread Looper = new Thread(new ThreadStart(SendFromQueueTrade));
                Looper.Start();

                Thread Looper1 = new Thread(new ThreadStart(SendFromQueuePrice));
                Looper1.Start();
                */

                _priceClient = new TcpClient(_host, _pricePort);
                _priceStreamSSL = new SslStream(_priceClient.GetStream(), false, new RemoteCertificateValidationCallback(ValidateServerCertificate), null);
                _priceStreamSSL.AuthenticateAsClient(_host);
                _tradeClient = new TcpClient(_host, _tradePort);
                _tradeStreamSSL = new SslStream(_tradeClient.GetStream(), false, new RemoteCertificateValidationCallback(ValidateServerCertificate), null);
                _tradeStreamSSL.AuthenticateAsClient(_host);
                _messageConstructor = new MessageConstructor(_host, _username, _password, _senderCompID, _senderSubID, _targetCompID);
                }catch(Exception e){
                    MessageBox.Show(e.ToString());
                }
        }


        // refresh from another thread
        public void RefreshTextBox(string value)
        {
            if (InvokeRequired)
            {
                this.Invoke(new Action<string>(RefreshTextBox), new object[] { value });
                return;
            }
            txtMessageReceived.Text += value;
        }

        private void SendFromQueueTrade()
        {
        while (true)
            {
            if (xQueueTrade.Count > 0)
                {
                string message = xQueueTrade.Dequeue().ToString();
                // txtMessageReceived.Text = SendPriceMessage(message);
                string msg = SendPriceMessage(message);
                // Gets executed on a seperate thread and doesn't block the UI while sleeping            
                RefreshTextBox(msg);
                }
            }
        }

        private void SendFromQueuePrice()
        {
        while (true)
            {
            if (xQueuePrice.Count > 0)
                {
                string message = xQueuePrice.Dequeue().ToString();
                // txtMessageReceived.Text = SendPriceMessage(message);
                string msg = SendPriceMessage(message);
                // Gets executed on a seperate thread and doesn't block the UI while sleeping            
                RefreshTextBox(msg);
                }
            }
        }

        private static bool ValidateServerCertificate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
        {
            if (sslPolicyErrors == SslPolicyErrors.None)
                return true;
            Console.WriteLine("Certificate error: {0}", sslPolicyErrors);
            return false;
        }

        private void ClearText()
        {
            txtMessageSend.Text = "";
            txtMessageReceived.Text = "";
        }

        private string SendPriceMessage(string message, bool readResponse = true)
        {
            xQueuePrice.Enqueue(message);
            string o = "";
            string msg = "";
            while (xQueuePrice.Count > 0)
            {
                msg = xQueuePrice.Dequeue().ToString();
                o = o + SendMessage(msg, _tradeStreamSSL, readResponse) + "<|>";
            }
            return o; 
        }

        private string SendTradeMessage(string message, bool readResponse = true)
        {
            xQueueTrade.Enqueue(message);
            string o = "";
            string msg = "";
            while (xQueueTrade.Count > 0)
            {
                msg = xQueueTrade.Dequeue().ToString();
                o = o + SendMessage(msg, _tradeStreamSSL, readResponse) + "<|>";
            }
            return o; 
        }

        private string SendMessage(string message, SslStream stream, bool readResponse = true)
        {
            try{
                var byteArray = Encoding.ASCII.GetBytes(message);
                stream.Write(byteArray, 0, byteArray.Length);
                var buffer = new byte[8128];
                if (readResponse)
                {
                    Thread.Sleep(100);
                    stream.Read(buffer, 0, 8128);
                }
                _messageSequenceNumber++;
                var returnMessage = Encoding.ASCII.GetString(buffer);
                return returnMessage;
            }catch(Exception e){
                // return e.ToString(); 
                return "[ERROR_SEND]";
            }
        }

        private void btnLogon_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.LogonMessage(MessageConstructor.SessionQualifier.QUOTE, _messageSequenceNumber, 30, false);
            txtMessageSend.Text = message;
            //xQueuePrice.Enqueue(message);
            txtMessageReceived.Text = SendPriceMessage(message);
        }

        private void btnTestRequest_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.TestRequestMessage(MessageConstructor.SessionQualifier.QUOTE, _messageSequenceNumber, _testRequestID);
            _testRequestID++;
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendPriceMessage(message);
        }

        private void btnLogout_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.LogoutMessage(MessageConstructor.SessionQualifier.QUOTE, _messageSequenceNumber);
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendPriceMessage(message);
            _messageSequenceNumber = 1;
        }

        private void btnMarketDataRequest_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.MarketDataRequestMessage(MessageConstructor.SessionQualifier.QUOTE, _messageSequenceNumber, "EURUSD:WDqsoT", 1, 0, 0, 1, 1);
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendPriceMessage(message);
        }

        private void btnHeartbeat_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.HeartbeatMessage(MessageConstructor.SessionQualifier.QUOTE, _messageSequenceNumber);
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendPriceMessage(message,false);
        }

        private void btnResendRequest_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.ResendMessage(MessageConstructor.SessionQualifier.QUOTE, _messageSequenceNumber, _messageSequenceNumber - 1);
            _testRequestID++;
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendPriceMessage(message);
        }

        private void btnReject_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.RejectMessage(MessageConstructor.SessionQualifier.QUOTE, _messageSequenceNumber, 0);
            _testRequestID++;
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendPriceMessage(message);
        }

        private void btnSequenceReset_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.SequenceResetMessage(MessageConstructor.SessionQualifier.QUOTE, _messageSequenceNumber, 0);
            _testRequestID++;
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendPriceMessage(message);
        }

        private void btnNewOrderSingle_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.NewOrderSingleMessage(MessageConstructor.SessionQualifier.TRADE, _messageSequenceNumber, "1408471", 1, 1, DateTime.UtcNow.ToString("yyyyMMdd-HH:mm:ss"), 1000, 1, "1");
            _testRequestID++;
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendTradeMessage(message);
        }

        private void btnOrderStatusRequest_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.OrderStatusRequest(MessageConstructor.SessionQualifier.TRADE, _messageSequenceNumber, "1408471");
            _testRequestID++;
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendTradeMessage(message);
        }

        private string Timestamp()
        {
            return (DateTime.UtcNow.Subtract(new DateTime(1970, 1, 1))).TotalSeconds.ToString();
        }

        private void btnRequestForPositions_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.RequestForPositions(MessageConstructor.SessionQualifier.TRADE, _messageSequenceNumber, "1408471", "");
            _testRequestID++;
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendTradeMessage(message);
        }

        private void btnLogonT_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.LogonMessage(MessageConstructor.SessionQualifier.TRADE, _messageSequenceNumber, 30, false);
            txtMessageSend.Text = message;
            // xQueueTrade.Enqueue(message);
            txtMessageReceived.Text = SendTradeMessage(message);            
        }

        private void btnHeartbeatT_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.HeartbeatMessage(MessageConstructor.SessionQualifier.TRADE, _messageSequenceNumber);
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendTradeMessage(message, false);
        }

        private void btnTestRequestT_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.TestRequestMessage(MessageConstructor.SessionQualifier.TRADE, _messageSequenceNumber, _testRequestID);
            _testRequestID++;
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendTradeMessage(message);
        }

        private void btnLogoutT_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.LogoutMessage(MessageConstructor.SessionQualifier.TRADE, _messageSequenceNumber);
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendTradeMessage(message);
            _messageSequenceNumber = 1;
        }

        private void btnResendRequestT_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.ResendMessage(MessageConstructor.SessionQualifier.TRADE, _messageSequenceNumber, _messageSequenceNumber - 1);
            _testRequestID++;
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendTradeMessage(message);
        }

        private void btnSpotMarketData_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.MarketDataRequestMessage(MessageConstructor.SessionQualifier.QUOTE, _messageSequenceNumber, "EURUSD:WDqsoT", 1, 1, 0, 1, 1);
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendPriceMessage(message);
        }

        private void btnStopOrder_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.NewOrderSingleMessage(MessageConstructor.SessionQualifier.TRADE, _messageSequenceNumber, "10", 1, 1, DateTime.UtcNow.ToString("yyyyMMdd-HH:mm:ss"), 1000, 3, "3", 0, (decimal)1.08);
            _testRequestID++;
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendTradeMessage(message);
        }

        private void btnLimitOrder_Click(object sender, EventArgs e)
        {
            ClearText();
            var message = _messageConstructor.NewOrderSingleMessage(MessageConstructor.SessionQualifier.TRADE, _messageSequenceNumber, "10", 1, 1, DateTime.UtcNow.ToString("yyyyMMdd-HH:mm:ss"), 1000, 2, "3", (decimal)1.08);
            _testRequestID++;
            txtMessageSend.Text = message;
            txtMessageReceived.Text = SendTradeMessage(message);
        }
    }
}

 


@mindbreaker
Replies

mindbreaker
30 Mar 2017, 22:32

RE:

there is mistake but it does not matter:

        private string SendPriceMessage(string message, bool readResponse = true)
        {
            xQueuePrice.Enqueue(message);
            string o = "";
            string msg = "";
            while (xQueuePrice.Count > 0)
            {
                msg = xQueuePrice.Dequeue().ToString();
                o = o + SendMessage(msg, _priceStreamSSL, readResponse) + "<|>";
            }
            return o; 
        }

I ask only about Trade part


@mindbreaker

Spotware
31 Mar 2017, 09:44

Hi mindbreaker,

In case you have multiple positions open, the server will stream multiple position reports. Therefore you will need to keep reading your stream for incoming data.


@Spotware

mindbreaker
31 Mar 2017, 12:12

RE:

Spotware said:

Hi mindbreaker,

In case you have multiple positions open, the server will stream multiple position reports. Therefore you will need to keep reading your stream for incoming data.

In prev  FIX API sample version  works fine and send all messages. What is wrong now (where is working example)?


@mindbreaker

Spotware
31 Mar 2017, 12:22

We have checked the example project and all positions are streamed without a problem. 


@Spotware

mindbreaker
31 Mar 2017, 15:18

RE:

Spotware said:

We have checked the example project and all positions are streamed without a problem. 

I do not think so (I test it on yours example).

When I send a query again I never get all opened positions.!

Something is wrong with this server.


@mindbreaker

Spotware
31 Mar 2017, 16:04

Try to wait more time before you read the buffer. 


@Spotware

mindbreaker
31 Mar 2017, 17:31

RE:

Thanks for the help.

I'm not interested anymore. I already have another solution

Ps. At this rate it will not work for the next 10 years.

Regards


@mindbreaker

#EOL
03 Apr 2017, 17:42

RE: RE:

mindbreaker said:

 

When I send a query again I never get all opened positions.!

Something is wrong with this server.

Just opened 1990 positions and got them all on the Request for positions. If there is an error it's not in the server.

You may want to refer TotalNumPosReports (tag 727) from the first position report to get know how many reports to expect


@#EOL

mindbreaker
04 Apr 2017, 21:54

RE: RE: RE:

#EOL said:

mindbreaker said:

 

When I send a query again I never get all opened positions.!

Something is wrong with this server.

Just opened 1990 positions and got them all on the Request for positions. If there is an error it's not in the server.

You may want to refer TotalNumPosReports (tag 727) from the first position report to get know how many reports to expect

Then why sometimes send all positions? It is server error!

I show everything from stream with MessageBox.Show()

I test it on two different servers and two different visual studio (2012 and 2010)

(i just download example from FIX-API-sample without changes)


@mindbreaker

mindbreaker
05 Apr 2017, 12:30

Maybe add movie with error to youtube? If you cant tell me what is wrong?


@mindbreaker

#EOL
05 Apr 2017, 12:46

RE: RE: RE: RE:

mindbreaker said:

Then why sometimes send all positions? It is server error!

I show everything from stream with MessageBox.Show()

I test it on two different servers and two different visual studio (2012 and 2010)

(i just download example from FIX-API-sample without changes)

Have you worked with FIX before?

Have you tried any third-party FIX clients other than FIX-API-sample or homebrew clients based on FIX-API-sample? (For example MiniFIX?)

Could you provide any FIX logs?

What do you have in tag 727 in the report?

How many positions do you expect?

When you receive your position partially, are all the FIX messages complete? (end with |10=[checksum])

        private string SendMessage(string message, SslStream stream, bool readResponse = true)
        {
            var byteArray = Encoding.ASCII.GetBytes(message);
            stream.Write(byteArray, 0, byteArray.Length);
            var buffer = new byte[1024];
            if (readResponse)
            {
                Thread.Sleep(100);
                stream.Read(buffer, 0, 1024);
            }
            _messageSequenceNumber++;
            var returnMessage = Encoding.ASCII.GetString(buffer);
            return returnMessage;
        }

I'm not a developer, but this looks ugly to me. Meaning that size of average FIX position report is 200 bytes, only 5.12 reports will be shown and I have no idea what will happen with the other. And this is true only in ideal zero-latency world, since it waits only for 100 ms from sending the request. What is you round-trip round-trip delay time to the server?

So I would not be trust to FIX-API-sample. It's a sample, not a production-ready solution.


@#EOL

mindbreaker
05 Apr 2017, 13:03

RE: RE: RE: RE: RE:

#EOL said:

mindbreaker said:

Then why sometimes send all positions? It is server error!

I show everything from stream with MessageBox.Show()

I test it on two different servers and two different visual studio (2012 and 2010)

(i just download example from FIX-API-sample without changes)

Have you worked with FIX before?

Have you tried any third-party FIX clients other than FIX-API-sample or homebrew clients based on FIX-API-sample? (For example MiniFIX?)

Could you provide any FIX logs?

What do you have in tag 727 in the report?

How many positions do you expect?

When you receive your position partially, are all the FIX messages complete? (end with |10=[checksum])

        private string SendMessage(string message, SslStream stream, bool readResponse = true)
        {
            var byteArray = Encoding.ASCII.GetBytes(message);
            stream.Write(byteArray, 0, byteArray.Length);
            var buffer = new byte[1024];
            if (readResponse)
            {
                Thread.Sleep(100);
                stream.Read(buffer, 0, 1024);
            }
            _messageSequenceNumber++;
            var returnMessage = Encoding.ASCII.GetString(buffer);
            return returnMessage;
        }

I'm not a developer, but this looks ugly to me. Meaning that size of average FIX position report is 200 bytes, only 5.12 reports will be shown and I have no idea what will happen with the other. And this is true only in ideal zero-latency world, since it waits only for 100 ms from sending the request. What is you round-trip round-trip delay time to the server?

So I would not be trust to FIX-API-sample. It's a sample, not a production-ready solution.

Hi i've got 727=4

And in my example buffer is 4096 but it send what want (why this does not display positions in json format !!!!!!????).

Ok it doesn't matter.

Thanks for the help.

 

 

 


@mindbreaker

#EOL
05 Apr 2017, 14:28

RE: RE: RE: RE: RE: RE:

mindbreaker said:

Hi i've got 727=4

And in my example buffer is 4096 but it send what want (why this does not display positions in json format !!!!!!????).

Ok it doesn't matter.

Thanks for the help.

Do you receive 4 position reports? Can you put a breakpoint to see the buffer? How many positions do you acually have and see in cTreader/cTrader Web?

Buffer size does not matter if the reports have not received yet. From my point of view, it should be separate thread that reads the stream, separates business messages (like execution reports, position reports, etc) from connection messages (like logon, heartbeats, test requests), stores the first ones to queue, raise specific events and maintains connection messages by itself. But implementing such framework would be re-inventing QuickFIX or similar engine.


@#EOL

mindbreaker
05 Apr 2017, 19:37

RE: RE: RE: RE: RE: RE: RE:

#EOL said:

mindbreaker said:

Hi i've got 727=4

And in my example buffer is 4096 but it send what want (why this does not display positions in json format !!!!!!????).

Ok it doesn't matter.

Thanks for the help.

Do you receive 4 position reports? Can you put a breakpoint to see the buffer? How many positions do you acually have and see in cTreader/cTrader Web?

Buffer size does not matter if the reports have not received yet. From my point of view, it should be separate thread that reads the stream, separates business messages (like execution reports, position reports, etc) from connection messages (like logon, heartbeats, test requests), stores the first ones to queue, raise specific events and maintains connection messages by itself. But implementing such framework would be re-inventing QuickFIX or similar engine.

But sometimes show  one positions sometimes 2 or somtimes all positions.

Is should return json string with all positions. Thats it.

Thanks


@mindbreaker

mindbreaker
05 Apr 2017, 19:39

RE: RE: RE: RE: RE: RE: RE: RE:

How many positions do you acually have and see in cTreader/cTrader Web?

 I have only one positions now.

Fuck it. It sith.


@mindbreaker

#EOL
06 Apr 2017, 11:04

RE: RE: RE: RE: RE: RE: RE: RE:

mindbreaker said:

But sometimes show  one positions sometimes 2 or somtimes all positions.

Is should return json string with all positions. Thats it.

I don't understand what json you talk about.

FIX is not json and json is never mentioned in your code.

If you have any doubts about the server -- check first with MiniFIX and if it works with it, it's not server fault, but your application


@#EOL

mindbreaker
06 Apr 2017, 11:43

RE: RE: RE: RE: RE: RE: RE: RE: RE:

#EOL said:

mindbreaker said:

But sometimes show  one positions sometimes 2 or somtimes all positions.

Is should return json string with all positions. Thats it.

I don't understand what json you talk about.

FIX is not json and json is never mentioned in your code.

If you have any doubts about the server -- check first with MiniFIX and if it works with it, it's not server fault, but your application

JSON string (https://pl.wikipedia.org/wiki/JSON) if you dont know what is it.

Positions should be recived with json (it simple idea)

Just testing an example from github! I do not use my code or application just this simple example.

Written by Spotware. Unfortunately it does not work as it should.

And I do not use frameworks.


@mindbreaker

#EOL
06 Apr 2017, 13:05

RE: RE: RE: RE: RE: RE: RE: RE: RE: RE:

Well, I know what json is and I whould like to assure you that there is nothing related to JSON in the example application, because it uses FIX protocol that was invented much much earlier

Related to the example -- i repeat, it's an exmple software, not a production-ready solution. In some cases it works, in some -- it does not. If you don't have certain skills in concurent software development, probably this is not your way.


@#EOL

mindbreaker
27 Jun 2017, 18:56

RE: RE: RE: RE: RE: RE: RE: RE: RE: RE: RE:
        static string ReadMessage(SslStream sslStream)
        {
            // Read the  message sent by the server.
            // The end of the message is signaled using the
            // "" marker.
            byte[] buffer = new byte[2048];
            StringBuilder messageData = new StringBuilder();
            int bytes = -1;
            do
            {
                bytes = sslStream.Read(buffer, 0, buffer.Length);

                // Use Decoder class to convert from bytes to UTF8
                // in case a character spans two buffers.
                Decoder decoder = Encoding.UTF8.GetDecoder();
                char[] chars = new char[decoder.GetCharCount(buffer, 0, bytes)];
                decoder.GetChars(buffer, 0, bytes, chars, 0);
                messageData.Append(chars);
                // Check for EOF.
                if (messageData.ToString().IndexOf("") != -1)
                {
                    break;
                }
            } while (bytes != 0);

            return messageData.ToString();
        }

 


@mindbreaker