Re: [hybi] websocket test suite

[Greg Wilkins <[e-mail]>]

Ahmed,

The autobahn test suite looks to be way ahead of where I got to.
My plan was to integrate jetty to autobahn tests and then review those
tests to see if they can be improved or not.

cheers





On 1 September 2011 07:33, Ahmed Soliman <[e-mail]> wrote:
> Greg,
>
> Are you still planning to add any of these scenarios to the jetty server?
>
> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On Behalf Of Ahmed Soliman
> Sent: Tuesday, August 09, 2011 2:45 AM
> To: Greg Wilkins; Hybi
> Subject: Re: [hybi] websocket test suite
>
> Greg,
>
> I have the following comments:
> 1. Scenario to add:
>        * Receive/send control frames (e.g. close frame) within a fragmented message. Verify that client/server responds with a close frame.
>
> 2. Focusing on interoperability testing:  Shouldn't we narrow the tests below to focus on interoperability testing?  I think some of the negative tests below  fit better in functional testing rather than interoperability testing? What do you think?
> 3. Standard process/test implementation: I think this will make it easier to share test implementations and verify interoperability between different protocol implementations. Any thoughts around this?
> (e.g. cre ating server endpoints per scenario for server implementation,  using sub-protocol to select between fragmented message/non-fragmented message, .... etc)
>
> Thanks,
> Ahmed
>
> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On Behalf Of Greg Wilkins
> Sent: Thursday, July 28, 2011 8:15 PM
> To: Hybi
> Subject: Re: [hybi] websocket test suite
>
> Here is a first cut at a list of tests for section 4 of the spec.  Are these the kinds of tests people were thinking about?
> If so, then I'm happy to go the next step of detail and start providing more detail for each test.
>
> Obviously some implementation are not going to be able to send some of the invalid frames or force fragmentation, so some suites will have to record some tests as skipped, or only testing in one direction etc.
>
>
>
>
> 4.2 Base Framing Tests
> ----------------------
>
> * 7 bit length: send/receive frames of length 0-125
> * 16 bit length: send/receive frames of length 126-128, 65536
> * 63 bit length: send/receive frames of length 65537
>
> * Non zero RSV: send frames with non zero reserved bits and no extension. Verify connection is failed.
>
> * Known Opcodes: send/receive frames with text,binary,ping,pong opcodes (continuation & close tested separately).
> * Unknown Opcodes: send frames with unknown opcodes 3-7, B-F. Verify connection is failed.
>
>
> 4.3 Client to Server Masking
> ----------------------------
>
> ???
>
>
> 4.4 Fragmentation
> -----------------
>
> * 1 fragment message: send/receive message in 1 fragment
> * 2 fragment message: send/receive message in 2 fragments
> * 3 fragment message: send/receive message in 3 fragments
> * Injected control: send/receive 2 fragment message with unsolicited pong message between frames
> * Fragmented control: send fragmented ping. Verify that connection is failed.
> * Interleaved fragments: send a fragment with FIN set and Continuation opcode. Verify that connection is failed.
> * Handled control: send/receive 2 fragment message with ping message between frames and last frame delayed. Verify pong is received before last frame is sent.
> * 7 bit fragment: send/receive message with initial/middle/final fragments of sizes 0-125
> * 16 bit fragment: send/receive message with initial/middle/final fragments of sizes 126-128,65536
> * 64 bit fragment: send/receive message with initial/middle/finalfragments of size 65537
>
>
> 4.5.1 Close Frames
> ------------------
> * Orderly Close: On an idle connection, send a close and receive a close.
> * Busy Close: send a close to an endpoint that has sent the first fragment of a message.
> * Simultaneous Close: send a close on both ends of a connection.
> * Message after close: attempt to send a message after a close. Verify message is not received.
> * Idle close: ??
> * Error Close: ??
>
>
> 4.5.2 Ping
> ----------
> * Empty ping: Send an empty ping, verify pong is received.
> * Non empty ping: Send a non empty ping, verify pong is received.
>
>
> 4.5.3 Pong
> ----------
> * Empty pong: Send an empty ping, verify pong is received and is empty.
> * Non empty pong: Send a non empty ping, verify pong is received and has matching content.
> * Outstanding ping: Send two pings with different content, reply with content for only the second. Verify connection stays open.
> * Unsolicited pong: Send/recv unsolicited pong.
>
>
> 4.5.4 Data Frames
> -----------------
> * UTF-8 Text 0000-007f: Send/receive text message with characters in code point range  U+0000 to U+007F
> * UTF-8 Text 0080-07ff: Send/receive text message with characters in code point range  U+0080 to U+07FF
> * UTF-8 Text 0800-ffff: Send/receive text message with characters in code point range  U+0800 to U+FFFF
> * UTF-8 Text 010000-10ffff: Send/receive text message with characters in code point range  U+010000 to U+10FFFF
> * Illegal bytes UTF-8: Send message with invalid UTF-8 byte sequence.
> Verify message is not received.
> * Illegal codes UTF-8: Send message with invalid UTF-8 code points.
> Verify message is not received.
> * Binary frames: Send/receive message containing illegal UTF-8 bytes sequences and code points as binary message.
> _______________________________________________
> hybi mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/hybi
>
> _______________________________________________
> hybi mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/hybi
>
>