[hybi] deflate-stream vs. frame-by-frame

"Bob Gezelter" <gezelter@rlgsc.com> Wed, 22 June 2011 13:01 UTC

Return-Path: <gezelter@rlgsc.com>
X-Original-To: hybi@ietfa.amsl.com
Delivered-To: hybi@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id 6487511E814B for <hybi@ietfa.amsl.com>; Wed, 22 Jun 2011 06:01:03 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.367
X-Spam-Status: No, score=-2.367 tagged_above=-999 required=5 tests=[AWL=0.232, BAYES_00=-2.599]
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id UG+e1ZTDi7hO for <hybi@ietfa.amsl.com>; Wed, 22 Jun 2011 06:01:02 -0700 (PDT)
Received: from smtpoutwbe05.prod.mesa1.secureserver.net (smtpoutwbe05.prod.mesa1.secureserver.net []) by ietfa.amsl.com (Postfix) with SMTP id 891BB11E8139 for <hybi@ietf.org>; Wed, 22 Jun 2011 06:01:02 -0700 (PDT)
Received: (qmail 29870 invoked from network); 22 Jun 2011 13:01:01 -0000
Received: from unknown (HELO localhost) ( by smtpoutwbe05.prod.mesa1.secureserver.net with SMTP; 22 Jun 2011 13:01:01 -0000
Received: (qmail 17351 invoked by uid 99); 22 Jun 2011 13:01:01 -0000
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="utf-8"
User-Agent: Web-Based Email 5.5.05
Message-Id: <20110622060058.ef1fc80126c74c6c202a919c41c7bb0b.24a5465d55.wbe@email03.secureserver.net>
From: "Bob Gezelter" <gezelter@rlgsc.com>
To: hybi@ietf.org
Date: Wed, 22 Jun 2011 06:00:58 -0700
Mime-Version: 1.0
Subject: [hybi] deflate-stream vs. frame-by-frame
X-BeenThere: hybi@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Server-Initiated HTTP <hybi.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/hybi>, <mailto:hybi-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/hybi>
List-Post: <mailto:hybi@ietf.org>
List-Help: <mailto:hybi-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/hybi>, <mailto:hybi-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 22 Jun 2011 13:01:03 -0000


With regards to:

>I don't think we should take the "mux" extension into account when
>discussing the deflate-stream extension. Compressing "mux" channels is a
>special case that should be covered by the "mux" extension specification.
>Combining all multiplexed streams into a single deflate channel is a good
>idea because it is actually very cheap memory wise, otherwise compressing
>each sub-channel with a separate DEFLATE stream could end up being very
>expensive memory wise. You can expect over 200KB of memory needed per each
>sub-channel, this basically kills the whole purpose of sub-channels which
>are actually supposed to decrease the load on the server side. Two-three
>thousands of compressed sub-channels would be enough to put the server on
>its knees.
>I consider the deflate-stream extension just as basic transport compression
>which is offered "out of the box" by a WebSocket implementation,
>irrespective of whether it supports multiplexing or not.

There are several issues that are co-mingled in this thread.

First, there is the question of compression and masking. Compressing
masked text is of dubious utility, efficacy, and efficiency. Masking
tends to randomize text, random text is a poor candidate for compression
(in some cases, attempting to compress random text can actually cause
the "compressed" text to be larger than the original text). Taken solely
on this basis, ANY compression should precede masking.

Second, John Tamplin's draft multiplexing proposal well notes that
multiplexing opens up possibilities for various traffic management
devices (e.g., de-aggregators) at the WebSocket protocol level. Thus, a
collection of sub-channels within a WebSocket connection may end up at
different destination nodes. If the entire stream is compressed as a
whole, this will require the entire stream to be de-compressed and then
the individual sub-channels re-compressed before ongoing transmission.
This realizes little benefit over compression at the sub-channel level
(further noting that there may indeed be compression done in any event
at lower levels of the protocol stack).

Lastly, compression on a sub-channel by sub-channel basis (generally
meaning per-frame basis) does not imply a need for 200KB of buffer space
per sub-channel. Rather, the amount of buffer space required per
sub-channel depends upon the frame-size and the compression scheme used.
Some compression schemes can function with minimal context, others may
need more. In any event, compression works best with a high degree of
correlation between successive data elements, thus there is a logical
division between the frames being transmitted within different
sub-channels to (presumably) different applications.

- Bob Gezelter, http://www.rlgsc.com