non-aggregated Depth of Market required

Created at 31 Jul 2014, 23:36
How’s your experience with the cTrader Platform?
Your feedback is crucial to cTrader's development. Please take a few seconds to share your opinion and help us improve your trading experience. Thanks!
olddirtypipster's avatar

olddirtypipster

Joined 18.04.2014

non-aggregated Depth of Market required
31 Jul 2014, 23:36


Hopefully a cTrader engineer will pick this up and provide me with a fix (bespoke or otherwise) for the issue I am experiencing with their platform.

Currently, the cTrader Depth of Market view is displaying completely aggregated price feeds. This is a problem.

Suppose there are three liquidity providers RBS, Reuters and BNP, all reporting the same price, but at different volumes. So for instance, this would be a test case scenario:

RBS:1.61288@3m; Reuters: 1.61288@1m and BNP: 1.61288@5m

By the time that this price reaches the end user, this is what they will see: 1.61288@9m; a price where the volume at that price represents the sum total of all the volume of each liquidity provider.

Although this may be convenient for some, it is definitely inconvenient for those of us who are attempting to combine feeds from several c Trader brokers in order to compile a semi-total market depth of the overall market. It is extremely problematic due to the fact that different brokers may be using one or more of the same liquidity provider, and by the end user combining the feeds, they will end up with an overlapping and severely skewed market depth picture.

So here is what I propose; That there be an option provided where non-aggregated prices be displayed in the market depth window. That is, in the above case, the end user is presented with this view:

1.61288@3m

1.61288@1m

1.61288@5m

Seeing three prices in this way will immediately let the end user know that there are three currently unidentified liquidity providers providing different trade sizes for this price.

In this way, if the end user then compared this feed to those provided by the two other brokers and they consistently see the same price at the same volume across several brokers,, that it could be concluded that participating brokers must be sharing the same liquidity provider.

They could then exclude all but one of these prices when combining the feed across all broker feeds for a final aggregation.

say that that this  hinges on the ability for the participating broker to be able to see the non-aggregated prices as shown above, my argument to this is that they already have this ability. They are simply not utilizing it. .

In order for the broker to receive these feeds in the first place, they would have had to initially send the following:

Market Data Request,Snapshot plus updates, full orderbook, full or incremental refresh, and AggregatedBook or Multiple entries per side per price.

In FIX message terminology, the message request sent to the liquidity provider by the brokerage side would take the form of: 262=V|263=1|264=0|265=0 or 1|266='Y' or 'N'
.

The root of this issue is Field 266 (AggregateBook or Multiple entries). I am suspecting that either the platform and hence the participating retail side broker has their code hardwired to send this the command 266='Y'.

In which case, the liquidity provider will naturally have no choice  but to respond by sending them back an aggregated price which they then deliver to the end user. in this case, the response read back from the liquidity provider will be: 1.61288@9m

On the otherhand, all that is required is for the trade platform to send the request 266=@N' (Multiple entries per side per price) and the responded fed back by the liquidity provider would be:

1.61288@3m

1.61288@1m

1.61288@5m

This is a very simple fix that will make a world of difference. All that would be required is a one byte change to the fix protocol Market Data message, and possibly an option on the GUI to switch between 266='Y' and 266='N'

Even if you do not want to wholesale make this change, could I at least have this done for me, as I really need non-aggregated data.

Either this, or provide me with a set of FIX  gateway credentials to your liquidity providers, this way I can bypass cTrader and stream in my own Depth of Market data using my own proprietary FIX engine that works quite well!

Here's hoping that someone picks this up and offers a solution.

Cheers!


@olddirtypipster
Replies

Spotware
04 Aug 2014, 17:59

We cannot provide an access to non-aggregated Depth of Market.


@Spotware

olddirtypipster
04 Aug 2014, 18:51

RE:

Spotware said:

We cannot provide an access to non-aggregated Depth of Market.

May I ask why?


@olddirtypipster

olddirtypipster
04 Aug 2014, 21:04

RE:

Spotware said:

We cannot provide an access to non-aggregated Depth of Market.

Is this based on policy, or is it a technological restriction?

On your website you state this:

We Believe:

  • Trader and broker interests can be aligned
  • Traders deserve execution transparency
  • Dealer intervention is rudimentary in retail FX
  • Scalability should not cost you a new license
  • FIX is the standard; bridges are a crutch

The last statement suggests that you use FIX protocol to access the Full Book.

This means that you can request for a non-aggregated book from your liquidity providers, who would all support this feature.

Your second statement implies that you goal is to achieve transparency. By not having your end user see the non-aggregated prices, you are completely going against this!

You first statement suggests that you are willing to work with your end users.

To this end, i am eager to know why tyou are not able to show non-aggregated prices.

Regards.

 

 


@olddirtypipster

Spotware
05 Aug 2014, 12:08

cServer cannot be connected to several liquidity providers directly. cServer connects to the aggregator which provides aggregated prices from its liquidity providers. 
Therefore we cannot provide non-aggregated market depth.


@Spotware

olddirtypipster
05 Aug 2014, 12:35

This is very peculiar to say the least!

I have established demo FIX connections to several aggregators in the past, combining the feeds from them. These included Currenex, Thompson Reuters, Integral and DeltaFlow, and even LMAX. They all have allowed non-aggregated market data requests (266='N') to be made.

Having the ability to call for non-aggregated price feeds is fundamental to accessing transparent market depth! To say that an aggregator will not allow you to accomplish this is strange to me. Maybe you should renegotiate with these entities, then? Why else would there be an option imbedded in the FIX protocol for this to be accomplished, if this feature is not supported by the vast majority of aggregators?

1) Are you currently using one or several aggregators?

2) If you are using several, is cTrader then further aggregating the market data from each individual aggregator, and displaying these to the end user?

3) If so, how do you ensure that there is no overlap in the market data that you receive when displaying this to the end user? Overlap when compiling previously aggregated data will undoubtedly result in biased data, skewed in favor of the the liquidity provider with the greatest multiplicity across each aggregator, unless you are then able to separate the individual LP's out.

This will naturally bring the quality of data being provided into question.

For this reason, if it were not possible to de-aggregate the data coming in from a given aggregator, then it would not be possible to combine aggregators into one comprehensive feed, as they may all be using one or more of the same liquidity provider feed that could not be separated out! This would render any aggregator not supporting 266-'N' incompatible with any other aggregator. It is highly unlikely that this is the case.

4) Could you make the aggregator(s) that cTrader is currently connected to public knowledge?

Regards,


@olddirtypipster

Spotware
05 Aug 2014, 14:33

You can find some of liquidity aggregators fully-integrated with Spotware there: http://www.spotware.com/partners/liquidity-aggregators.

We have checked specifications of couple of aggregators. Indeed it is possible to retrieve non-aggregated market depths. However we do not plan to provide such functionality. You can post your idea to vote.spotware.com. We will consider to provide such functionality if it collects enough votes.


@Spotware

olddirtypipster
05 Aug 2014, 17:19

RE:

Spotware said:

You can find some of liquidity aggregators fully-integrated with Spotware there: http://www.spotware.com/partners/liquidity-aggregators.

We have checked specifications of couple of aggregators. Indeed it is possible to retrieve non-aggregated market depths. However we do not plan to provide such functionality. You can post your idea to vote.spotware.com. We will consider to provide such functionality if it collects enough votes.

Indeed, these are truly some high  powered aggregators with very reliable currency feeds! It is truly a pity that you are not allowing your users full access to their data in a way that would allow your platform to raise the bar on what can be done, setting a precedence never achieved by any other company before!

Now that it has been confirmed that non-aggregated market depth can be achieved, is there a reason why you do not plan to provide such functionality?

In order to achieve this, it would be quite literally an exceedingly small change as the major infrastructure to accomplish this has already been provided within your platform. You are simply not utilizing it.

Here are the steps to achieving this:

1) Change the FIX message market data request from a 266='Y' to a 266='N'

2) Provide a checkbox on the GUI that will toggle between the two states.

This could easily be done in the next release, so it would be very helpful if you could provide a reason why you are reluctant to accomplish this. Especially when this change will invariably add to the richness of your already well designed platform. It would also bring you much closer to the tenets that you originally stipulated here:

We Believe:

  • Trader and broker interests can be aligned
  • Traders deserve execution transparency
  • Dealer intervention is rudimentary in retail FIX
  • Scalability should not cost you a new license
  • FIX is the standard; bridges are a crutch

You are correct that I should put this to a vote, and I already have. However, there exists the fear that the difference between aggregated and non-aggregated market depth, thought very important, may very well be outside the scope of most low tiered traders, and it is likely that these are the ones populating this forum, who are mostly active seeking help seeking help in building cBots for their latest trading strategy.

The correct use of Depth of Market to help a trader acquire order-flow, helping them make informed decisions about the relationship between their trading, is a skill that only the most seasoned of traders would have acquired.

To this end, I have some very legitimate fears regarding the success of this voting process.

The majority of these experienced retail traders would simply not be using cTrader, as they would easily be able to afford the minimum deposits (Dukascopy for example requires a minimum £100,000 deposit or minimum volumes traded per month of around £500 million per month) to acquire a direct FIX gateway connection directly to the market. They would therefore not require cTrader to access the market depth as they would have their own proprietary platforms that allow them Direct Market Access.

It is more likely that the traders that do populate this forum are low tiered, and simply do not have sufficient knowledge about market depth and its importance to make an informed decision about non-aggregated data implementation! As such, I predict that my request may never reach the number of votes that you are looking for, not because most people here would not appreciate the added option, but would simply not care enough to cast their vote!

My recommendation is therefore at a severe disadvantage due to the very small numbers of informed traders who also use this forum.

This is a change that I could very easily accomplish on my own. If you are reluctant to make these changes wholesale, then at least arm me with the ability to make these changes on my own.

You see, cTrader represents one of the last few hopes for those experienced retail traders with limited investment capital, to access the rich data that exists to carry out their fundamental market analysis. Without it, we very literally have nothing else to fall back on!

In this case, I truly hope that there is something you can do.

I look forward to your response.


@olddirtypipster

Spotware
05 Aug 2014, 17:47

Now that it has been confirmed that non-aggregated market depth can be achieved, is there a reason why you do not plan to provide such functionality?

We do not plan to provide such functionality because it is first request to implement it. We are not sure that traders will find such functionality useful.


@Spotware

olddirtypipster
05 Aug 2014, 18:11

Firstly, as was stated previously,, the ability to view aggregated verses non-aggregated depth of market could be provided as an option. There is very strong evidence to support the fact non-aggregated data is fundamentally useful in general within the wider FOREX trading community. You really do not have to carry out your own census to determine how useful viewing non-aggregated market depth is, just as how you do  not necessarily have to carry out a census to determine how useful the MACD is to traders!

Unlike the MACD, however, "The analysis and interpretation of non-aggregated market depth in obtaining knowledge of market order-flow "is a far more advanced topic that that of "Analyzing the MACD to inform ones trading decisions".

Traders having a vested interest in having my recommended feature implemented would undoubtedly be far and few between, as this represents a more advanced item that would likely never be used by less experienced traders!

To address your curiosity about whether this feature will be used my most cTrader end users, the evidence (obtained by reviewing other posts made in these forums) suggests that it is not likely to be a hotly used feature! I can safely say this because there have been very few high level discussions about proper use of depth of market (DOM) on this board, and those that do exist were as a result of the OP's seeking clarity in using DOM.

Sadly therefore, it is unlikely that this request will ever acquire sufficient votes to draw your attention, simply because there are too few users who even understand what it means!

Now, if your mission statement is to provide functionality for the lowest common denominator, then you simply do not need to implement this feature at all, since the lowest common denominator will simply have no use for it. However, in doing so, you run the risk of detracting those persons seeking a a serious, open ended. intuitive trading platform.

Please note that adding this feature will in no way take away from the trading platform. It will only add to it a feature that, in the eyes of experienced traders, is glaringly absent!


@olddirtypipster

olddirtypipster
12 Aug 2014, 17:27 ( Updated at: 21 Dec 2023, 09:20 )

RE:

Case in point:

 

Here, you are seeing the non-aggregated display non-aggregated market depth of a broker's feed for GBPUSD.

Your platform could potentially outshine theirs, since yours is open ended. It would offer programmers to stream in this non-aggregated data to their statistical analysis toolbox, and gain a better edge on market orderflow.

Everyone benefits!

Spotware said:

Now that it has been confirmed that non-aggregated market depth can be achieved, is there a reason why you do not plan to provide such functionality?

We do not plan to provide such functionality because it is first request to implement it. We are not sure that traders will find such functionality useful.

 


@olddirtypipster

olddirtypipster
12 Aug 2014, 17:28 ( Updated at: 21 Dec 2023, 09:20 )

RE: RE:

olddirtypipster said:

Case in point:

 

Here, you are seeing the non-aggregated display non-aggregated market depth of a broker's feed for GBPUSD.

Your platform could potentially outshine theirs, since yours is open ended. It would offer programmers to stream in this non-aggregated data to their statistical analysis toolbox, and gain a better edge on market orderflow.

Everyone benefits!

Spotware said:

Now that it has been confirmed that non-aggregated market depth can be achieved, is there a reason why you do not plan to provide such functionality?

We do not plan to provide such functionality because it is first request to implement it. We are not sure that traders will find such functionality useful.

 

 


@olddirtypipster

Spotware
15 Sep 2014, 09:58

Thank you for your suggestion. We will consider it.


@Spotware

olddirtypipster
15 Sep 2014, 20:13

RE:

Spotware said:

Thank you for your suggestion. We will consider it.

Brilliant!

 

This is good news indeed.


@olddirtypipster

cjdduarte
27 Mar 2015, 18:29

News about this?


@cjdduarte

olddirtypipster
01 Apr 2015, 12:45

RE:

comment was removed by moderator


@olddirtypipster

Spotware
02 Apr 2015, 17:49

We still do not plan to provide this functionality.


@Spotware

olddirtypipster
02 Apr 2015, 18:33

I figured as much.

 

Well... I am a skilled programmer, and can do this for you. Would you consider?


@olddirtypipster

olddirtypipster
02 Apr 2015, 18:44

I am inclined to suspect that the reason why you adamantly do not want to add this most basic of features, is because other than the top of book price, the other price/volumes in the DOM view are algorithmically synthesized by the cTrader engine, and are not in-fact being streamed from LPs. Other than the top of book price, these prices are meaningless noise.


@olddirtypipster

olddirtypipster
02 Apr 2015, 20:46

This is what your platform should have to make it truly competitive.

http://www.youtube.com/watch?v=Kf-3FSe8-rY&t=6m31s

You are making a mistake by not considering it.


@olddirtypipster

Spotware
06 Apr 2015, 14:20

I am inclined to suspect that the reason why you adamantly do not want to add this most basic of features, is because other than the top of book price, the other price/volumes in the DOM view are algorithmically synthesized by the cTrader engine, and are not in-fact being streamed from LPs.

We do not generate DoM. DoM is taken from LP directly.


@Spotware

olddirtypipster
26 Jun 2015, 23:08

RE:

Spotware said:

I am inclined to suspect that the reason why you adamantly do not want to add this most basic of features, is because other than the top of book price, the other price/volumes in the DOM view are algorithmically synthesized by the cTrader engine, and are not in-fact being streamed from LPs.

We do not generate DoM. DoM is taken from LP directly.

I want to be able to differentiate between the LP  prices displayed.

I understand that you cannot divulge the identities of some.all of your LP's, but what you can do is provide a means to view all prices provided by the IP's instead of automatically aggregating them.


@olddirtypipster

Spotware
29 Jun 2015, 12:22

Dear Trader,

Thank you for your suggestion. We will consider it. Additionally you can post your ideas/suggestions to http://vote.spotware.com/


@Spotware

olddirtypipster
29 Jun 2015, 23:58

RE:

Spotware said:

Dear Trader,

Thank you for your suggestion. We will consider it. Additionally you can post your ideas/suggestions to http://vote.spotware.com/

I keep getting mixec messages. On one hand, one of you will say you refuse to do this. Then, on the other hand, someone else says we will consider it.

I would like to see this feature implemented in the next release. I have been speaking on this for over a year now.


@olddirtypipster

Spotware
30 Jun 2015, 11:19

Dear Trader,

Since you asked again for access to non-aggregated Depth of Market, we will consider it.


@Spotware

olddirtypipster
30 Jun 2015, 20:22

RE:

Spotware said:

Dear Trader,

Since you asked again for access to non-aggregated Depth of Market, we will consider it.

This gives me a glimmer of hope. Here's hoping to see it in the next release!


@olddirtypipster

MerlinBrasil
19 Jul 2015, 08:58

Non-Aggregated Depth of Market

Dear Spotware,

I'm also EXTREMELY INTERESTED in having this functionality.

Thanks for your persistence in the face of very mixed messages, ODPipster, and thanks for that link.

Actual real-time view of the LPs book is critical and since you even offered to code it for them, I can not understand why Spotware has been so adament in their refusal to consider it.

Not having RT non-aggregated DOM is much like trading blindfolded. Having that data puts Retail Traders in a position only the Big Dogs currently enjoy.

PLEASE reconsider your position. ODPipster has been unnaturally patient. I can't imagine why an honest broker (ok, no laughing.....) would have any problem demonstrating this kind of transparency and transparency is what Spotware says they're all about, yes?

Thanks,

Merlin (SkypeID: merlinbrasil)
 


@MerlinBrasil

olddirtypipster
19 Jul 2015, 20:43

RE: Non-Aggregated Depth of Market

MerlinBrasil said:

Dear Spotware,

I'm also EXTREMELY INTERESTED in having this functionality.

Thanks for your persistence in the face of very mixed messages, ODPipster, and thanks for that link.

Actual real-time view of the LPs book is critical and since you even offered to code it for them, I can not understand why Spotware has been so adament in their refusal to consider it.

Not having RT non-aggregated DOM is much like trading blindfolded. Having that data puts Retail Traders in a position only the Big Dogs currently enjoy.

PLEASE reconsider your position. ODPipster has been unnaturally patient. I can't imagine why an honest broker (ok, no laughing.....) would have any problem demonstrating this kind of transparency and transparency is what Spotware says they're all about, yes?

Thanks,

Merlin (SkypeID: merlinbrasil)
 

Quite frankly, the only reason why a provider would not make non-aggregated market feed unavailable is if the level II price stream is faked.

It would be very easy to identify illegitimate (synthesized) non-aggregated price feeds by comparing the individual LP prices for a given range to those produced from another source.

For instance, if the broker purports that two of their LP's are Barclays and BNP, then you'd simply archive a snippet of these prices for a specific time, then call up these two banks and ask for their prices for the same timeline.

If the prices match, then you'd know that the DOM is legit. If they do not match up, then you'd know that the DOM is spewing garbage data with zero correlation to the real market.

Because the individual LP prices are aggregated, there is no way of comparing them, and a broker could be hiding their dishonesty.

For example, it allows a broker the wiggle room to engineer internal spikes in their pricing feed, that do not necessarily occur in the direct market. It would be impossible for the average user to verify these spikes. There are several othr ways a broker could get away with internalized price manipulation by hiding behind the fact that they stream aggregated price feeds.


@olddirtypipster