Protocol message tag had invalid wire type

Spotware
22 Mar 2017, 10:46

Hi Taharxeg,

There should be something wrong with your code which is not obvious from the code sample you sent. We advise you to cross check your implementation with our code samples which are working properly https://github.com/spotware

taharxeg
27 Mar 2017, 02:16

Hi,

I tried both your C# and nodejs examples, and it worked fine, but I still get the same error message from C++ and Python, I am not sure if I am doing this right.

Here is the Python code.

from OpenApiMessages_pb2 import *
from OpenApiModelMessages_pb2 import *
import struct


autReq = ProtoOAAuthReq()
autReq.payloadType = OA_AUTH_REQ
autReq.clientSecret = "49p1ynqfy7c4sw84gwoogwwsk8cocg8ow8gc8o80c0ws448cs4"
autReq.clientId = "7_5az7pj935owsss8kgokcco84wc8osk0g0gksow0ow4s4ocwwgc"

pack_2 = autReq.SerializeToString()
pack_1 = struct.pack('>I', len( pack_2 )  ) # the first part of the message is length

fullPacket = pack_1 + pack_2
print( fullPacket )

# Get hexadecimal bytes
bytes = " ".join("{:02x}".format(ord(c)) for c in fullPacket )
print( bytes.upper() )

When I tried the C# code the right serialized message was like this ( in Hex ):

0000006F08D00F126A1234375F35617A37706A3933356F77737373386B676F6B63636F38347763386F736B306730676B736F77306F773473346F63777767631A3234397031796E7166793763347377383467776F6F677777736B38636F6367386F77386763386F383063307773343438637334

But my C++ and Python code generates a message like this:

0000006D08D00F1234375F35617A37706A3933356F77737373386B676F6B63636F38347763386F736B306730676B736F77306F773473346F63777767631A3234397031796E7166793763347377383467776F6F677777736B38636F6367386F77386763386F383063307773343438637334

You can see that the message bytes are almost identical except the 14 first bytes.

It would be great if you can take the time to check this.

honeybadger
15 Nov 2020, 23:46

Workaround

Hi taharxeg and Spotware team,

I encountered the same problem and have literally just now found a solution/workaround as described here.

.

0. The Code Snippet in question (Python):

proto = OAMessages.ProtoOAApplicationAuthReq()
# following line is equal to proto.payloadType = 2100
proto.payloadType = int(OAMessagesModel.ProtoOAPayloadType.PROTO_OA_APPLICATION_AUTH_REQ)
proto.clientId = self.CID
proto.clientSecret = self.CSEC

### debug message
print(proto.serializeToString())
### end debug

# transmits the message to the server; tx(msg) adds a uint32 LENGTH message before
self.tx(proto.serializeToString())

.

1. The Problem

The generated Protobuf classes yield a different ByteString in Python vs. C#. This is true for the ProtoOAAuthReq class but perhaps for other classes, too. @taharxeg posted the two different ByteStrings, and I can reproduce this behavior with my own Client ID and Client Secret (both strings are the pure messages without the separate uint32 LENGTH message at the beginning):

C# ByteString (as in Spotware's C# sample project) represented in Hex:

08 B4 10 12 6D 12 37 [Client ID] 1A 32 [Client Secret]

Python ByteString represented in Hex:

08 B4 10 12 37 [Client ID] 1A 32 [Client Secret]

Problem Description: The Python-generated ByteString is missing 2 bytes between its 3rd and 4th byte.

.

2. The Result

As a result, sending my Python ProtoMessage yields the response:

\x08\xde\x10\x12>\x1a\x0fINVALID_REQUEST"+Protocol message tag had invalid wire type.\x1a2[Client Secret]

.

3. Error Tracing

Upon investigation of the Spotware example project, I find the following behavior in the file Open Api Library/OpenApiMessagesFactory.cs, lines 173-179:

public ProtoMessage CreateAppAuthorizationRequest(string clientId, string clientSecret, string clientMsgId = null)
{
  var _msg = ProtoOAApplicationAuthReq.CreateBuilder();
  _msg.SetClientId(clientId);
  _msg.SetClientSecret(clientSecret);
  return CreateMessage((uint)_msg.PayloadType, _msg.Build().ToByteString(), clientMsgId);
}

To me, the return statement looks like a generic ProtoMessage is being built that contains the the three fields (uint32) payloadType, (byte[]) payload, and (string) clientMsgId. The payload field, in contrast, contains the ByteString of a ProtoOAApplicationAuthReq message with the format (string) clientId, (string) clientSecret. This seems to be different than building a ProtoOAAuthReq with Python.

.

4. Workaround (EDITED)

This is not optimized, I just tried to confirm my suspicion - and there you go! The workaround is to mimic this behavior. Obviously the field values are preceded by a 2-byte "field indicator" or similar, so I cut off the first 3 bytes of my Python ByteString and embed the remainder in a generic ProtoMessage:

proto = OAMessages.ProtoOAApplicationAuthReq()
proto.clientId = self.CID
proto.clientSecret = self.CSEC
msg = bytearray(proto.SerializeToString())

wrapper = OACommon.ProtoMessage()
wrapper.payloadType = OAMessagesModel.ProtoOAPayloadType.PROTO_OA_APPLICATION_AUTH_REQ
wrapper.payload = bytes(msg[3:])
# EDIT: this works also with the original ProtoOAApplicationAuthReq, i.e.
# wrapper.payload = proto.SerializeToString()

#### debug messages
print("ProtoMessage:\n")
print(wrapper.SerializeToString())
#### end debug

self.tx(wrapper.SerializeToString())

Result: And suddenly, the server understands me! 

\x08\xde\x10\x12:\x1a\x16CH_CLIENT_AUTH_FAILURE" OA client is not in active state

(This error message is expected b/c my app has not been activated for the API yet).

.

5. The bug

Although I am not sure, the behavior of Google's/Python's ProtoOAApplicationAuthReq class looks unwanted to me. Judging from the Open API Reference, the server expects a generic ProtoMessage with the format (uint32) payloadType | (byte[]) payload | (string) clientMsgId. In my opinion, ProtoOAApplicationAuthReq.SerializeToString() should either

• wrap the payload in a ProtoMessage and add the payloadType in the ProtoMessage but not in the ProtoOAApplicationAuthReq message, or
• create the payload only - without the payloadType - and leave the payload ByteString for manual wrapping, as in the C# example project. (EDIT: After identifying The Solution - see below - it turns out this is exactly what the method does.)

I am not sure what exactly the desired behavior should be (maybe even the C# implementation has "undesired" behavior in this sense?) - and I hope you can shed some light on this. Certainly the desired behavior should not be to generate a ByteString first and then manipulate it manually (cut off the first 3 bytes) before continuing. If the OpenApiMessagesFactory.cs was automatically generated by ProtoGen or similar, this smells like a bug on Google's side - this is why the perspective of the Spotware team on this is so valuable.

.

EDIT: 6. The Solution

I just found out that the string manipulation is not necessary. So the correct procedure is 

1. Generate ProtoOAApplicationAuthReq message
2. Serialize message to ByteString
3. Generate generic ProtoMessage wrapper
   • Set payloadType according to the message in (1)
   • Set payload to the serialized message from (2)
   • Optionally set clientMsgId
4. Do what you like with your awesome, server-readable ProtoMessage!

After understanding this, I do see that the Open API reference somehow captures this, but I still think that the way it is described in the Reference is somewhat confusing to us amateur programmers. It would be amazing if you could update the reference to clarify the necessary extra step and the difference between the Open API Common Messages (the "message wrappers") and the Open API Messages (the payload of a ProtoMessage).

.

Thanks for taking the time to check this!

Best,

h.

amusleh
16 Dec 2021, 16:05

Hi,

For anyone who is reading this thread in future, you can use our new Python package for cTrader open API: OpenApiPy (spotware.github.io)