Re: [hybi] CML

[John Tamplin <[e-mail]>]

On Mon, Aug 23, 2010 at 12:00 AM, Thomson, Martin <[e-mail]
> wrote:

> > There were objections to fragmentation without it...
>
> Where do you expect to see CML included?  The first frame, every frame, the
> last frame, or some other pattern?
>

Where the fragmentation is introduced by an intermediary or by a sender with
fixed-size but large content, I would expect it to be only on the first
frame and it would be completely accurate.  If a sender is fragmenting
dynamically generated content, I would expect an estimate might be included
on the first frame, and then an accurate count might be included once when
it is first known, if it is known before the last frame.  The estimate could
conceivably be updated several times if a more accurate estimate became
available.  In the worst case, the receiver still would have to realloc/copy
the buffer the same number of times if it had no CML estimates, but an
argument was made that it could be better and that it enabled UI affordances
like progress meters.


> Presumably, much of the benefit is lost if the complete length is only
> reported on the last frame - that information could be trivially gained by
> doing a little arithmetic (and I hear that computers are good at that ;).
>  That leads me to conclude that it’s most useful on the first frame.  Or at
> least as soon as possible.
>

Correct.


> ISTM that the most of the other constraints will come from the structure of
> the API.  If you need to pass a single array, you might suffer.  Otherwise,
> a simple abstraction (e.g. Java's InputStream or Reader classes, maybe
> ByteBuffer depending on paradigm) can help you avoid needing to re-alloc().
>

Remember that the current JS API is message based, where it receives a
single string containing the entire message.  It seems likely browsers are
going to allocate an entire buffer for the incoming message (granted some of
them will want that to be in UTF16 characters, which won't be tied to the
number of UTF8 bytes - there was discussion of perhaps adding that as an
extension), filling it as it reads fragments, and then handing it off to
client code once the last frame is received.


> If the receiver is so constrained that they can't deal with multiple
> frames, how able are they to use a BigInteger effectively, which would be
> necessary to hold CML?  Speaking from the Java perspective, this would be
> relatively easy, though not exactly efficient.
>

A 63-bit integer is a Java long, no BigInteger required (the high bit was
specifically required to be 0 to support Java's lack of an unsigned 64-bit
type).


> Overall, I'm not seeing a strong case for including CML.  It seems
> relatively harmless, but harmlessness is not really great motivation.
>

Primarily I suggested it as a way to avoid the apparently endless arguments
we were having about fragmentation.


> p.s. I still struggle to identify what an intermediary is able to do with
> this protocol.
>

Initially, I think of proxies as the likely intermediary, and I see two
roles for them:

   - enforce corporate policies, such as restricting which sites can be
   contacted, sniff content for malware or corporate IP being leaked, etc
   - aggregate connections - a MUX extension would be helpful here to reduce
   the number of outbound connections when many internal users are hitting the
   same sites

Also, an intermediary might be used as a front end to a server farm, which
receives the connection, aggregates them, and load-balances to backend
servers.

-- 
John A. Tamplin
Software Engineer (GWT), Google