Re: [hybi] Framing Take VI (a compromise proposal)

[Scott Ferguson <[e-mail]>]

John Tamplin wrote:
> On Sat, Aug 14, 2010 at 5:09 AM, Anne van Kesteren <[e-mail] 
> <mailto:[e-mail]>> wrote:
>
>     On Fri, 13 Aug 2010 23:16:14 +0200, Ian Hickson <[e-mail]
>     <mailto:[e-mail]>> wrote:
>
>         2010/8/12 Ian Fette (イアンフェッティ) <[e-mail]
>         <mailto:[e-mail]>>:
>
>
>             That said, here's the proposal:
>
>
>         This seems overly complex. [...]
>
>
>     Agreed. I would really like to avoid having so much features in
>     the initial deployment.
>
>
> [The following is quick-and-dirty code I wrote on the spot, so don't 
> pick it apart -- it is just to illustrate the point that it isn't that 
> complicated]
>
> For a simple sender which doesn't negotiate any extensions and sends 
> only text in unfragmented frames, it looks like this:
>
> Connection::sendMessage(char* utf8, int utf8len) {

Well, the complexity is more of a spec issue than a coding one because 
none of these proposals are hugely complicated to implement. 
(Implementing them efficiently is a different matter.)

A Jamie-style proposal less complex because it leaves the basic frame 
clean without needing the extension length or the extension payload and 
without the extra complete-length field. The frame would look like:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+---------+-------------------------------+---------------+
|I|F|B|opcode(5)| length(16)                    |               |
|N|I|I|         |                               |               |
|I|N|G|         |                               |               |
+-+-+-+---------+-------------------------------+ - - - - - - - +
| big-length(48) if BIG = 1                                     |
+ - - - - - - - +-----------------------------------------------+
|               | payload(length)                               |
+---------------+                                               |
|                                                               |
+---------------------------------------------------------------+

As a measure of complexity, compare the features that can vary the 
frame-header size:

  a) short/long payload length
  b) extension data
  c) short/long extension length
  d) complete message-length

A Jamie-style extension only has the single complexity-feature (a) to 
the frame-header, while your proposal adds 3 more (b,c,d). Since the 
Jamie-style extension supports all the extension capabilities that yours 
does, there's no added capabilities with the extra complexity added by 
b,c,d.

As a separate issue, your code's signature doesn't capture the essence 
of the problem, because you really need to consider all the APIs when 
evaluating a framing proposal, not just one that happens to be easy to 
implement for the proposal. Your example was a specialized API that 
really only works for utf-8 encoded 8-bit C-strings.

The basic sender APIs that framing needs to support are:

a)  sendTextMessage(char []buffer, int offset, int length); // 
JavaScript 16-bit text send
a*) sendTextMessage(byte []buffer, int offset, int length); // utf-8 
encoded text (like PHP)

b)  sendBinaryMessage(byte []buffer, int offset, int length); // binary 
single-buffer send

c)  class BinaryMessageOutputStream {
    write(byte []buffer, int offset, int length);
    close();
  }

d)  class TextMessageWriter {
    write(char []buffer, int offset, int length);
    close();
  }

The receiver variations are

a) char []receiveTextMessage()

b) byte []receiveBinaryMessage() // if returning utf-8 8-bit strings, 
this is the text receive

c) class BinaryMessageInputStream {
      int read(byte []buffer, int offset, int length);
    }

d) class TextMessageReader {
      int read(char []buffer, int offset, int length);
    }

A good frame design will support all 5 sender API styles and 4 receiver 
API styles in a straightforward/efficient manner. You can't just pick 
one API that happens to make a frame proposal work.

-- Scott

>   writeByte(0xC0 | TEXT_FRAME);
>   if (utf8len >= 127) {
>     writeByte(127);
>     writeUnsignedLong(utf8len);
>   } else {
>     writeByte(utf8len);
>   }
>   writeBytes(utf8, utf8len);
> }
>
> For a receiver, there is additional complication because of the need 
> to support fragmented messages, but that has been accepted as a 
> requirement.  It isn't much worse:
>
> Connection::readFrame) {
>   int frameType = getByte();
>   int opcode = frameType & 0xF;
>   if (opcode != TEXT_FRAME) // error, discard, whatever
>   long length = getByte();
>   if (length == 127) {
>     length = getUnsignedLong();
>   }
>   switch (frameType & 0xC0) {
>      case 0x80: // initial frame of fragmented message
>        if (state == IN_MESSAGE) // error
>        state = IN_MESSAGE;
>        int messageLength = getUnsignedLong();
>        bufLen = messageLength ? messageLength : DEFAULT_BUFFER_SIZE;
>        buf = new char[bufLen];
>        bufIdx = 0;
>        // fallthrough
>      case 0x00: // intermediate frame of fragmented message
>        if (state != IN_MESSAGE) // error
>        growBufferIfNeeded(length);
>        getBytes(buf + bufIdx, length);
>        bufIdx += length;
>        break;
>      case 0x40: // final frame of fragmented message
>        if (state != IN_MESSAGE) // error
>        // fallthrough
>      case 0xC0: // unfragmented message
>        growBufferIfNeeded(length);
>        getBytes(buf + bufIdx, length);
>        deliverUtf8TextMessage(buf, bufIdx + length);  // receiver is 
> responsible for freeing
>        state = NOT_IN_MESSAGE;
>        buf = 0;
>        bufLen = 0;
>        break;
>    }
> }
>
> Connection::growBufferIfNeeded(int length) {
>    if (buf && bufLen >= bufIdx + length) return;
>    int newBufLen = (bufLen == 0) ? length : bufLen * 2;
>    char* newbuf = new char[newBufLen];
>    if (buf) {
>      memcpy(newbuf, buf, bufIdx);
>      delete[] buf;
>    }
>    buf = newBuf;
>    bufLen = newBufLen;
> }
>     
> Given the requirement for fragmentation support, it isn't clear how 
> you make it a lot simpler.  Note that the parts dealing with the frame 
> structure are just as simple as in the sender case.
>
> -- 
> John A. Tamplin
> Software Engineer (GWT), Google
> ------------------------------------------------------------------------
>
> _______________________________________________
> hybi mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/hybi
>