Autobahn wstest 12.4.* help?!?!

Greg_Wilkins · January 4, 2018, 2:34pm

Hi,

I’m having trouble with passing the Autobahn wstests 12.4.* I can see why I’m failing, but I’m not sure what I should actually be doing.

The 12.4.* tests involve sending a large HTML file (which includes UTF-8 characters) in slices that are sent with per-message compress.

I’m failing the tests where the slices breakup a mutli-byte UTF-8 character.

For example, in test 12.4.5, all is good until frame 869, which arrives and I receive parse it as follows:

2018-01-04 15:22:47.016:DBUG:oejwcP.SERVER:qtp1304836502-17: Parsing DirectByteBuffer@171a7a53[p=0,l=1019,c=4096,r=1019]={<<<\xC1\xFe\x03\xF3\xC0p\x87Q\x1c’J?\x1b6\x97\xEf7…\xC0\x90a\xB2$\xAdj\xA4\x1dd;\xEb\xD6\x8e\x86>>>\x84$Kr\ \xC6\x87v\xDbzE\xEe\xF1\x98\x0cF…\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}

2018-01-04 15:22:47.016:DBUG:oejwcP.SERVER:qtp1304836502-17: OpCode TEXT, fin=true rsv=1…

2018-01-04 15:22:47.016:DBUG:oejwcP.SERVER:qtp1304836502-17: Raw Payload: DirectByteBuffer@9d6b2e3[p=0,l=1011,c=1011,r=1011]={<<<\x1c’J?\x1b6\x97\xEf7Y\xA1\xBb\x01y\xA7\x1a\xE2…\xC0\x90a\xB2$\xAdj\xA4\x1dd;\xEb\xD6\x8e\x86>>>}

2018-01-04 15:22:47.016:DBUG:oejwcP.SERVER:qtp1304836502-17: Parsed Frame: TEXT@726feedd[len=1011,fin=true,rsv=100,masked=true] : DirectByteBuffer@171a7a53[p=1019,l=1019,c=4096,r=0]={\xC1\xFe\x03\xF3\xC0p\x87Q\xDcW\xCdn\xDbF\x10\xBe\xF7…\x00\xE0\xE6\xE3\xE4\xDd\xEd\xF5\xDd\x14\xBc\xBa\x16\xFe\x01<<<>>>\x84$Kr\ \xC6\x87v\xDbzE\xEe\xF1\x98\x0cF…\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}

2018-01-04 15:22:47.016:DBUG:oejwci.WebSocketConnection:qtp1304836502-17: onFrame(TEXT@726feedd[len=1011,fin=true,rsv=100,masked=true])

So it is a masked TEXT frame, fin==true with rsv bit 1 set indicating it is compressed. I unmask and uncompress:

2018-01-04 15:22:47.016:DBUG:oejwcec.CompressExtension:qtp1304836502-17: Decompressed 4096 bytes: Inflater[finished=false,read=401627,written=3563520,remaining=0,in=401627,out=3563520]

2018-01-04 15:22:47.016:DBUG:oejwcec.CompressExtension:qtp1304836502-17: Supplied 4 input bytes: Inflater[finished=false,read=401627,written=3563520,remaining=4,in=401627,out=3563520]

2018-01-04 15:22:47.016:DBUG:oejwcec.CompressExtension:qtp1304836502-17: Decompress: read 0 Inflater[finished=false,read=401631,written=3563520,remaining=0,in=401631,out=3563520]

2018-01-04 15:22:47.016:DBUG:oejwcec.CompressExtension:qtp1304836502-17: Decompress: exiting Inflater[finished=false,read=401631,written=3563520,remaining=0,in=401631,out=3563520]

2018-01-04 15:22:47.016:DBUG:oejwcec.PerMessageDeflateExtension:qtp1304836502-17: nextIncomingFrame(TEXT@7d2e365f[len=4096,fin=true,rsv=000,masked=true])

2018-01-04 15:22:47.016:DBUG:oejwc.WebSocketChannel:qtp1304836502-17: incomingFrame(TEXT@7d2e365f[len=4096,fin=true,rsv=000,masked=true], org.eclipse.jetty.websocket.core.io.WebSocketConnection$1@513e8f4e) - connectionState=WebSocketChannelState[OPEN{i=true o=true c=-1}], handler=org.eclipse.jetty.websocket.core.autobahn.AutobahnFrameHandler@7315adaa

2018-01-04 15:22:47.016:DBUG:oejwc.AbstractFrameHandler:qtp1304836502-17: TEXT: HeapByteBuffer@336dd9f1[p=0,l=4096,c=4096,r=4096]={<<<Case Descripti…T 1:
Hello-\xC2>>>}

The frame expands to 4096 bytes as expected, It contains the expected HTML, but ends with 0xC2, which is the first byte of a two byte UTF-8 character. Looking at the source file, this is a valid slice of that file in bytes.

But this is a fin==true text frame, so I have to convert the payload to a string to pass to my echo handler. Thus I’m detecting that the UTF-8 is incomplete and throw an exception, which makes me fail the test?!?!?

I can’t see how I’m meant to handle that incomplete UTF-8? If the frame was fin==false, then we correctly handle such partial characters, but the fin=true tells me that there is no more parts of this message and it looks to me to really be a bad TEXT message?!?!?

I’m using this is using Autobahn 0.10.9 AutobahnTestSuite 0.7.6 and we are passomg all the other tests

What am I missing?

cheers

Greg_Wilkins1 · January 10, 2018, 9:56am

Anybody??

Matt_Bonneau · January 11, 2018, 1:49am

If the message is FIN=true, it is a text frame, and it ends in the middle of a multi-byte character, then the message is not valid. I am guessing there is something else going on.

Is this an open-source project? Could you give some links to the code that you are testing?

Matt

···

On Thursday, January 4, 2018 at 9:34:52 AM UTC-5, Greg Wilkins wrote:

Hi,

I’m having trouble with passing the Autobahn wstests 12.4.* I can see why I’m failing, but I’m not sure what I should actually be doing.

The 12.4.* tests involve sending a large HTML file (which includes UTF-8 characters) in slices that are sent with per-message compress.

I’m failing the tests where the slices breakup a mutli-byte UTF-8 character.

For example, in test 12.4.5, all is good until frame 869, which arrives and I receive parse it as follows:

2018-01-04 15:22:47.016:DBUG:oejwcP.SERVER:qtp1304836502-17: Parsing DirectByteBuffer@171a7a53[p=0,l=1019,c=4096,r=1019]={<<<\xC1\xFe\x03\xF3\xC0p\x87Q\x1c’J?\x1b6\x97\xEf7…\xC0\x90a\xB2$\xAdj\xA4\x1dd;\xEb\xD6\x8e\x86>>>\x84$Kr\ \xC6\x87v\xDbzE\xEe\xF1\x98\x0cF…\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}

2018-01-04 15:22:47.016:DBUG:oejwcP.SERVER:qtp1304836502-17: OpCode TEXT, fin=true rsv=1…

2018-01-04 15:22:47.016:DBUG:oejwcP.SERVER:qtp1304836502-17: Raw Payload: DirectByteBuffer@9d6b2e3[p=0,l=1011,c=1011,r=1011]={<<<\x1c’J?\x1b6\x97\xEf7Y\xA1\xBb\x01y\xA7\x1a\xE2…\xC0\x90a\xB2$\xAdj\xA4\x1dd;\xEb\xD6\x8e\x86>>>}

2018-01-04 15:22:47.016:DBUG:oejwcP.SERVER:qtp1304836502-17: Parsed Frame: TEXT@726feedd[len=1011,fin=true,rsv=100,masked=true] : DirectByteBuffer@171a7a53[p=1019,l=1019,c=4096,r=0]={\xC1\xFe\x03\xF3\xC0p\x87Q\xDcW\xCdn\xDbF\x10\xBe\xF7…\x00\xE0\xE6\xE3\xE4\xDd\xEd\xF5\xDd\x14\xBc\xBa\x16\xFe\x01<<<>>>\x84$Kr\ \xC6\x87v\xDbzE\xEe\xF1\x98\x0cF…\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}

2018-01-04 15:22:47.016:DBUG:oejwci.WebSocketConnection:qtp1304836502-17: onFrame(TEXT@726feedd[len=1011,fin=true,rsv=100,masked=true])

So it is a masked TEXT frame, fin==true with rsv bit 1 set indicating it is compressed. I unmask and uncompress:

2018-01-04 15:22:47.016:DBUG:oejwcec.CompressExtension:qtp1304836502-17: Decompressed 4096 bytes: Inflater[finished=false,read=401627,written=3563520,remaining=0,in=401627,out=3563520]

2018-01-04 15:22:47.016:DBUG:oejwcec.CompressExtension:qtp1304836502-17: Supplied 4 input bytes: Inflater[finished=false,read=401627,written=3563520,remaining=4,in=401627,out=3563520]

2018-01-04 15:22:47.016:DBUG:oejwcec.CompressExtension:qtp1304836502-17: Decompress: read 0 Inflater[finished=false,read=401631,written=3563520,remaining=0,in=401631,out=3563520]

2018-01-04 15:22:47.016:DBUG:oejwcec.CompressExtension:qtp1304836502-17: Decompress: exiting Inflater[finished=false,read=401631,written=3563520,remaining=0,in=401631,out=3563520]

2018-01-04 15:22:47.016:DBUG:oejwcec.PerMessageDeflateExtension:qtp1304836502-17: nextIncomingFrame(TEXT@7d2e365f[len=4096,fin=true,rsv=000,masked=true])

2018-01-04 15:22:47.016:DBUG:oejwc.WebSocketChannel:qtp1304836502-17: incomingFrame(TEXT@7d2e365f[len=4096,fin=true,rsv=000,masked=true], org.eclipse.jetty.websocket.core.io.WebSocketConnection$1@513e8f4e) - connectionState=WebSocketChannelState[OPEN{i=true o=true c=-1}], handler=org.eclipse.jetty.websocket.core.autobahn.AutobahnFrameHandler@7315adaa

2018-01-04 15:22:47.016:DBUG:oejwc.AbstractFrameHandler:qtp1304836502-17: TEXT: HeapByteBuffer@336dd9f1[p=0,l=4096,c=4096,r=4096]={<<<Case Descripti…T 1:
Hello-\xC2>>>}

The frame expands to 4096 bytes as expected, It contains the expected HTML, but ends with 0xC2, which is the first byte of a two byte UTF-8 character. Looking at the source file, this is a valid slice of that file in bytes.

But this is a fin==true text frame, so I have to convert the payload to a string to pass to my echo handler. Thus I’m detecting that the UTF-8 is incomplete and throw an exception, which makes me fail the test?!?!?

I can’t see how I’m meant to handle that incomplete UTF-8? If the frame was fin==false, then we correctly handle such partial characters, but the fin=true tells me that there is no more parts of this message and it looks to me to really be a bad TEXT message?!?!?

I’m using this is using Autobahn 0.10.9 AutobahnTestSuite 0.7.6 and we are passomg all the other tests

What am I missing?

cheers

Greg_Wilkins1 · January 11, 2018, 8:32am

If the message is FIN=true, it is a text frame, and it ends in the middle of a multi-byte character, then the message is not valid. I am guessing there is something else going on.

That’s my interpretation… yet autobahn has been testing impls for a while now and unless this test has been recently changed, I doubt it is a wrong test… however that is how the test is described - slicing a large HTML document into 4096 byte chunks which are then compressed and sent as text messages?

Is this an open-source project? Could you give some links to the code that you are testing?

It is the jetty-server. We have done a major refactor of our core websocket implementation and this test is failing.

You can see the code here: https://github.com/eclipse/jetty.project/tree/jetty-10.0.x-websocket/jetty-websocket/websocket-core

regards

···

On Thursday, 11 January 2018 02:49:32 UTC+1, Matt Bonneau wrote:

Matt

On Thursday, January 4, 2018 at 9:34:52 AM UTC-5, Greg Wilkins wrote:

Hi,

I’m having trouble with passing the Autobahn wstests 12.4.* I can see why I’m failing, but I’m not sure what I should actually be doing.

The 12.4.* tests involve sending a large HTML file (which includes UTF-8 characters) in slices that are sent with per-message compress.

I’m failing the tests where the slices breakup a mutli-byte UTF-8 character.

For example, in test 12.4.5, all is good until frame 869, which arrives and I receive parse it as follows:

2018-01-04 15:22:47.016:DBUG:oejwcP.SERVER:qtp1304836502-17: Parsing DirectByteBuffer@171a7a53[p=0,l=1019,c=4096,r=1019]={<<<\xC1\xFe\x03\xF3\xC0p\x87Q\x1c’J?\x1b6\x97\xEf7…\xC0\x90a\xB2$\xAdj\xA4\x1dd;\xEb\xD6\x8e\x86>>>\x84$Kr\ \xC6\x87v\xDbzE\xEe\xF1\x98\x0cF…\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}

2018-01-04 15:22:47.016:DBUG:oejwcP.SERVER:qtp1304836502-17: OpCode TEXT, fin=true rsv=1…

2018-01-04 15:22:47.016:DBUG:oejwcP.SERVER:qtp1304836502-17: Raw Payload: DirectByteBuffer@9d6b2e3[p=0,l=1011,c=1011,r=1011]={<<<\x1c’J?\x1b6\x97\xEf7Y\xA1\xBb\x01y\xA7\x1a\xE2…\xC0\x90a\xB2$\xAdj\xA4\x1dd;\xEb\xD6\x8e\x86>>>}

2018-01-04 15:22:47.016:DBUG:oejwcP.SERVER:qtp1304836502-17: Parsed Frame: TEXT@726feedd[len=1011,fin=true,rsv=100,masked=true] : DirectByteBuffer@171a7a53[p=1019,l=1019,c=4096,r=0]={\xC1\xFe\x03\xF3\xC0p\x87Q\xDcW\xCdn\xDbF\x10\xBe\xF7…\x00\xE0\xE6\xE3\xE4\xDd\xEd\xF5\xDd\x14\xBc\xBa\x16\xFe\x01<<<>>>\x84$Kr\ \xC6\x87v\xDbzE\xEe\xF1\x98\x0cF…\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}

2018-01-04 15:22:47.016:DBUG:oejwci.WebSocketConnection:qtp1304836502-17: onFrame(TEXT@726feedd[len=1011,fin=true,rsv=100,masked=true])

So it is a masked TEXT frame, fin==true with rsv bit 1 set indicating it is compressed. I unmask and uncompress:

2018-01-04 15:22:47.016:DBUG:oejwcec.CompressExtension:qtp1304836502-17: Decompressed 4096 bytes: Inflater[finished=false,read=401627,written=3563520,remaining=0,in=401627,out=3563520]

2018-01-04 15:22:47.016:DBUG:oejwcec.CompressExtension:qtp1304836502-17: Supplied 4 input bytes: Inflater[finished=false,read=401627,written=3563520,remaining=4,in=401627,out=3563520]

2018-01-04 15:22:47.016:DBUG:oejwcec.CompressExtension:qtp1304836502-17: Decompress: read 0 Inflater[finished=false,read=401631,written=3563520,remaining=0,in=401631,out=3563520]

2018-01-04 15:22:47.016:DBUG:oejwcec.CompressExtension:qtp1304836502-17: Decompress: exiting Inflater[finished=false,read=401631,written=3563520,remaining=0,in=401631,out=3563520]

2018-01-04 15:22:47.016:DBUG:oejwcec.PerMessageDeflateExtension:qtp1304836502-17: nextIncomingFrame(TEXT@7d2e365f[len=4096,fin=true,rsv=000,masked=true])

2018-01-04 15:22:47.016:DBUG:oejwc.WebSocketChannel:qtp1304836502-17: incomingFrame(TEXT@7d2e365f[len=4096,fin=true,rsv=000,masked=true], org.eclipse.jetty.websocket.core.io.WebSocketConnection$1@513e8f4e) - connectionState=WebSocketChannelState[OPEN{i=true o=true c=-1}], handler=org.eclipse.jetty.websocket.core.autobahn.AutobahnFrameHandler@7315adaa

2018-01-04 15:22:47.016:DBUG:oejwc.AbstractFrameHandler:qtp1304836502-17: TEXT: HeapByteBuffer@336dd9f1[p=0,l=4096,c=4096,r=4096]={<<<Case Descripti…T 1:
Hello-\xC2>>>}

The frame expands to 4096 bytes as expected, It contains the expected HTML, but ends with 0xC2, which is the first byte of a two byte UTF-8 character. Looking at the source file, this is a valid slice of that file in bytes.

But this is a fin==true text frame, so I have to convert the payload to a string to pass to my echo handler. Thus I’m detecting that the UTF-8 is incomplete and throw an exception, which makes me fail the test?!?!?

I can’t see how I’m meant to handle that incomplete UTF-8? If the frame was fin==false, then we correctly handle such partial characters, but the fin=true tells me that there is no more parts of this message and it looks to me to really be a bad TEXT message?!?!?

I’m using this is using Autobahn 0.10.9 AutobahnTestSuite 0.7.6 and we are passomg all the other tests

What am I missing?

cheers

Greg_Wilkins1 · January 11, 2018, 8:50am

Also if anybody can point me to some code that does pass this test, then I’ll have a look to see what it is doing differently.

Greg_Wilkins · January 17, 2018, 10:37am

So I’ve not yet found a server that passes tests 12.4.*

wstest --mode echoserver does not implement permessage-deflate
wstest --mode testeeserver fails the test the same way as my code
jetty-9.4.x, which passed the full test suite when it was written fails in the same way.

I’m concluding that 12.4.* is a bad test! I can’t see anyway it can actually be implemented other than treating the frames as binary, which is not really passing the test.

Matt_Bonneau · January 17, 2018, 3:09pm

Hi Greg,

This branch of the Ratchet/RFC6455 library passes 12.4.*: https://github.com/ratchetphp/RFC6455/tree/permessage-deflate

Let me know if you would like instructions on running the tests and I can write some up.

I am using:

autobahn (0.10.9)

autobahntestsuite (0.7.5)

If I get a chance later today, I will try it with 0.7.6 to be at the same version you are at.

···

On Wednesday, January 17, 2018 at 5:37:55 AM UTC-5, Greg Wilkins wrote:

So I’ve not yet found a server that passes tests 12.4.*

wstest --mode echoserver does not implement permessage-deflate

wstest --mode testeeserver fails the test the same way as my code

jetty-9.4.x, which passed the full test suite when it was written fails in the same way.

I’m concluding that 12.4.* is a bad test! I can’t see anyway it can actually be implemented other than treating the frames as binary, which is not really passing the test.

Greg_Wilkins1 · January 17, 2018, 6:03pm

Matt,

thanks for that. I’ve not yet had a chance to try and run it, but I’ve had a look at the code and I think the echo function used is this bit, pasted below:

    function (MessageInterface $message, MessageBuffer $messageBuffer) use ($response) {

        $messageBuffer->sendMessage($message->getPayload(), true, $message->isBinary());

    },

My PHP is pretty much non existent, so can you tell me what the $message->getPayload() part does? Is that getting the payload as bytes or is it getting it as a string. If it is echoing the payload as bytes, then that would explain why it is passing without encountering the partial UTF-8 character. I assume that it is bytes as this code appears to be used for both text and binary messages.

So if my assumption is right, then I still think this is a bad test. It is sending text frames that cannot actually be interpreted as text, only as bytes.

If my assumption is wrong, how is getPayload handling the partial utf-8 character?

regards

···

On Wednesday, 17 January 2018 16:09:47 UTC+1, Matt Bonneau wrote:

Hi Greg,

This branch of the Ratchet/RFC6455 library passes 12.4.*: https://github.com/ratchetphp/RFC6455/tree/permessage-deflate

Let me know if you would like instructions on running the tests and I can write some up.

I am using:

autobahn (0.10.9)

autobahntestsuite (0.7.5)

If I get a chance later today, I will try it with 0.7.6 to be at the same version you are at.

On Wednesday, January 17, 2018 at 5:37:55 AM UTC-5, Greg Wilkins wrote:

So I’ve not yet found a server that passes tests 12.4.*

wstest --mode echoserver does not implement permessage-deflate

wstest --mode testeeserver fails the test the same way as my code

jetty-9.4.x, which passed the full test suite when it was written fails in the same way.

I’m concluding that 12.4.* is a bad test! I can’t see anyway it can actually be implemented other than treating the frames as binary, which is not really passing the test.

Matt_Bonneau · January 18, 2018, 4:38am

Greg,

Updated autobahntestsuite to 0.7.6 and the RFC6544 library now complains about invalid UTF-8 and “fails” the test. Looks like something changed between the two versions.

You are not alone: https://github.com/crossbario/autobahn-testsuite/issues/77

@oberstet recommends using the dockerized version until someone gets to fixing this: https://github.com/crossbario/crossbar-docker/tree/master/autobahn-testsuite

Regards

···

On Wednesday, January 17, 2018 at 1:03:04 PM UTC-5, Greg Wilkins wrote:

Matt,

thanks for that. I’ve not yet had a chance to try and run it, but I’ve had a look at the code and I think the echo function used is this bit, pasted below:

    function (MessageInterface $message, MessageBuffer $messageBuffer) use ($response) {

        $messageBuffer->sendMessage($message->getPayload(), true, $message->isBinary());

},

My PHP is pretty much non existent, so can you tell me what the $message->getPayload() part does? Is that getting the payload as bytes or is it getting it as a string. If it is echoing the payload as bytes, then that would explain why it is passing without encountering the partial UTF-8 character. I assume that it is bytes as this code appears to be used for both text and binary messages.

So if my assumption is right, then I still think this is a bad test. It is sending text frames that cannot actually be interpreted as text, only as bytes.

If my assumption is wrong, how is getPayload handling the partial utf-8 character?

regards

On Wednesday, 17 January 2018 16:09:47 UTC+1, Matt Bonneau wrote:

Hi Greg,

This branch of the Ratchet/RFC6455 library passes 12.4.*: https://github.com/ratchetphp/RFC6455/tree/permessage-deflate

Let me know if you would like instructions on running the tests and I can write some up.

I am using:

autobahn (0.10.9)

autobahntestsuite (0.7.5)

If I get a chance later today, I will try it with 0.7.6 to be at the same version you are at.

On Wednesday, January 17, 2018 at 5:37:55 AM UTC-5, Greg Wilkins wrote:

So I’ve not yet found a server that passes tests 12.4.*

wstest --mode echoserver does not implement permessage-deflate

wstest --mode testeeserver fails the test the same way as my code

jetty-9.4.x, which passed the full test suite when it was written fails in the same way.

I’m concluding that 12.4.* is a bad test! I can’t see anyway it can actually be implemented other than treating the frames as binary, which is not really passing the test.

Greg_Wilkins1 · January 18, 2018, 8:07am

Matt,

thanks for that. No idea why I was unable to find issue 77 myself… I did look… I just didn’t see!

cheers