Re: [hybi] how do we move forward on agreeing on framing?

[John Tamplin <[e-mail]>]

On Thu, Aug 19, 2010 at 3:43 AM, Dave Cridland <[e-mail]> wrote:

> I'd feel more comfortable with knowing that the extension framing (not
> actual support) was done. Otherwise, any extension mechanism is going to
> smell like a real hack, even if it's possible. I also worry about actors
> seeing extensions and choking on them, whether that's intermediaries or
> whatever.
>

The problem is we seem pretty strongly split between two different extension
methods:

   - having extension metadata separate from the payload, with some defined
   structure so that non-critical unknown extensions can be ignored.  This
   would also open the possibility of negotiating fixed offsets into the
   extension area for certain extension data for performance reasons
   - using nested layers of opcodes to define extensions -- ie the top-level
   opcode would be MUX, and the payload would have the mux data and the
   next-layer opcode and data, which might be another extension, etc, until you
   finally get to the real opcode/data.

The former gives the possibility of identifying the extension data where
intermediaries can safely ignore extensions they don't understand.  This is
not a proposal for extension data, but just a thought of how it might work
out to illustrate this point:

   - define one of the reserved bits as an EXT bit - if set, an extension
   length follows the header before the payload data, and that length is
   followed by a block of extension data
   - Each extensions data is preceded by an extension header
   - 1 bit - extension is critical - frame cannot be manipulated if this bit
      is set and the receiver does not understand the extension (and
an ultimate
      endpoint would drop the frame if it didn't understand the
extension, though
      something would be wrong since it negotiated it)
      - 7 bits - extension id - defined in a registry or negotiated during
      the handshake
      - 8 bits - extension data length (maybe support variable length if we
      think we need more than 255 bytes per extension)
      - extension data

This would allow a receiver to find the extension data it is looking for
even if it doesn't understand all the extensions.  Certainly some extensions
would render the data uninterpretable if they are not understood, but I
think many would not.  For example, an extension including a MIME-type for
the payload could safely be ignored, as could an extension that gave the
count of UTF16 characters.

The nested opcode approach essentially requires all outer extensions to be
understood, since otherwise you don't even know where in the payload the
next layer is.  It has the advantage of taking less space to define the
extension data and basically means there is no global extension format --
every extension defines its own payload data format.  Finally, the extension
data can be intermixed with lower-level payloads which may be easier to
handle different nesting of extensions.  For example, if you have a MUX
extension that aggregates multiple logical frames into a single frame, if
each of those frames have different extensions, it is less obvious how to
encode that into the aggregated frame, so that is an advantage of the
nested-opcode approach.

So, this split is why I gave a rationale that we don't have time to resolve
how extensions should be represented for the base protocol.  If you really
think it is critical that it is resolved now, then I suppose we have to do
it now, but I was hoping to basically encode what we have agreement on now
into the base protocol while leaving hooks for whatever we ultimately
decide.


> Finally, it's not clear to me if extension data needs to be per-message or
> per-frame - I can't think of a per-frame use-case, but still, this effects
> possible extension mechanisms heavily.
>

Clearly, if you have the ability to attach extension data to each frame, you
can define in the extension that it is present only on initial or final
frames as appropriate.  Regarding the per-frame use case, Ian has already
mentioned the mux stream id case.

-- 
John A. Tamplin
Software Engineer (GWT), Google