Re: Transfer-Encoding chunked: preserved in the wild?

Greg Wilkins <gregw@webtide.com> Mon, 25 January 2021 10:26 UTC

Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 287143A0E1B for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Mon, 25 Jan 2021 02:26:07 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.77
X-Spam-Level:
X-Spam-Status: No, score=-0.77 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001, MAILING_LIST_MULTI=-1, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=webtide-com.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1FujKAJyYM_w for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Mon, 25 Jan 2021 02:26:05 -0800 (PST)
Received: from lyra.w3.org (lyra.w3.org [128.30.52.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 560233A0E1A for <httpbisa-archive-bis2Juki@lists.ietf.org>; Mon, 25 Jan 2021 02:26:05 -0800 (PST)
Received: from lists by lyra.w3.org with local (Exim 4.92) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1l3z1p-0003sE-4y for ietf-http-wg-dist@listhub.w3.org; Mon, 25 Jan 2021 10:23:25 +0000
Resent-Date: Mon, 25 Jan 2021 10:23:25 +0000
Resent-Message-Id: <E1l3z1p-0003sE-4y@lyra.w3.org>
Received: from titan.w3.org ([128.30.52.76]) by lyra.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <gregw@webtide.com>) id 1l3z1n-0003rS-TJ for ietf-http-wg@listhub.w3.org; Mon, 25 Jan 2021 10:23:24 +0000
Received: from mail-ot1-x334.google.com ([2607:f8b0:4864:20::334]) by titan.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from <gregw@webtide.com>) id 1l3z1l-0008LN-VJ for ietf-http-wg@w3.org; Mon, 25 Jan 2021 10:23:23 +0000
Received: by mail-ot1-x334.google.com with SMTP id d1so12219702otl.13 for <ietf-http-wg@w3.org>; Mon, 25 Jan 2021 02:23:21 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=webtide-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Qytt5UbL5a4aBFFeZY+a5PBsRzFAr580WniO6P2nMYw=; b=v0jk1zONUswMZOrhi6Van0I7yv25fyyFiEDwox1a+7MwegBnVNsMu3vd9jhVPMrzHk NpV/Me/JsRXcCo0c4IhzqgVMn1TFopDiaqdn6bzln06xzhuSwimahG+Klsy2obkqNQb9 SyXXVTrm0UdC1AaPWrqjvz+HnrFIOUKWtEdFgF42NjK2lsODe7iOdQsp1vWjtBvLHLHV JaKK8W/GsEhzAiq2THiyV3LDntXryhVrAKjY+fvCOavafGPlYKnXn6WYVhR3+JhhJTiK LLzM7HmYzxP8BdCdNb8je7BsJ+7gOpDmOKg8s/mJJgEqx3FADnFclzbV1WksU4fr71gp iI7Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Qytt5UbL5a4aBFFeZY+a5PBsRzFAr580WniO6P2nMYw=; b=gKobYdAsYZea5WKaKIqNmrJivalklVC8HJHYREgHB0eTMJ4KMPMSf0guJmBEOP4mbF 2pE+jTxCiPD8cuInrsV8JLcaAg3mfp7XHT3aAi7V1hZuRK9O0kxrfPmdAM6xSuBJYsA0 6UIhJhzwtAwiIb+b2fAsLDOioTyy/e8kRXmRrz9CNpD+vEqCDU2Fs5JbmV5UWRHE7lpg dAv35/+6Z1dSrNOtgIWJ7f5d4/GS936cAV39iOTsURBm/d3cxzayl7jvU80DF+kgpt5j 99OK5zcaS/wYONNNsVtftHy9aKws1Zu2Ly/TdFIAgiIkGA8rXiIfC72DpkGGZZFOjYHa WCog==
X-Gm-Message-State: AOAM533dn3wodf/URzDOdpDgq8ZwUfPq0dLMjjZoKaw3H6tEO5/0M9AU WGmpux5Ecw0jIShu6w4YzY5PQ0N0nJYEfFIhqXgNiQ9X2U0=
X-Google-Smtp-Source: ABdhPJxW9xlLT3XkW7eyZBk3J4YHtNC6u+W0/gilw7NUJIjm9QQqao3U/EgiHkfJtdq0KbRP5BTXILmKv5uW/MSaxCo=
X-Received: by 2002:a05:6830:1c68:: with SMTP id s8mr11556771otg.192.1611570191058; Mon, 25 Jan 2021 02:23:11 -0800 (PST)
MIME-Version: 1.0
References: <b27f7ecd-348c-4ea7-941c-f3151476934c@www.fastmail.com>
In-Reply-To: <b27f7ecd-348c-4ea7-941c-f3151476934c@www.fastmail.com>
From: Greg Wilkins <gregw@webtide.com>
Date: Mon, 25 Jan 2021 11:23:00 +0100
Message-ID: <CAAPGdfH4XWzvVLXcaMuhuZR=VRsfr87TbO+HgqtQcwwLi4kh0Q@mail.gmail.com>
To: Seph Gentle <me@josephg.com>
Cc: HTTP Working Group <ietf-http-wg@w3.org>
Content-Type: multipart/alternative; boundary="000000000000c4dc1f05b9b6ec53"
Received-SPF: pass client-ip=2607:f8b0:4864:20::334; envelope-from=gregw@webtide.com; helo=mail-ot1-x334.google.com
X-W3C-Hub-Spam-Status: No, score=-3.9
X-W3C-Hub-Spam-Report: BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, W3C_AA=-1, W3C_WL=-1
X-W3C-Scan-Sig: titan.w3.org 1l3z1l-0008LN-VJ b26a2a30683c84a296609c486926e6d5
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Transfer-Encoding chunked: preserved in the wild?
Archived-At: <https://www.w3.org/mid/CAAPGdfH4XWzvVLXcaMuhuZR=VRsfr87TbO+HgqtQcwwLi4kh0Q@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/38391
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <https://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

I also do not think this is a workable solution.  Various middleman's
implemented with Jetty do not preserve chunk boundaries. Buffering and
processing may add extra chunks, amalgamate chunks or even replace them
with a content-length.   Also chunking is not applicable to HTTP >= 2.

There is server sent events... but not sure how well that is supported in
the wild.  We implement it, but have very little evidence of it being used.

Websocket or long polling are your best bet for multiple sending events
server to client.

cheers





On Mon, 25 Jan 2021 at 11:06, Seph Gentle <me@josephg.com> wrote:

> Hi everyone!
>
> I’m working with Mike and others to figure out & clean up the protocol for
> braid. We want to add real-time subscriptions to http.
>
> For this we need to send a stream of messages (values and patches) in
> response to a single http request. The simplest way to encode that would be
> to lean on transfer-encoding: chunked and wrap each patch in exactly one
> http “chunk”, so we don’t need to do our own message framing.
>
> I want to lean on some collective wisdom here. Is this a bad idea? Does
> anyone know if middleman proxy servers ever move chunk boundaries around?
> Is that valid according to the protocol? Is that something we should worry
> about?
>
> (Server sent events do their own message framing on top of the transfer
> encoding. Is there a good reason for that?).
>
> -Seph
>
>
>

-- 
Greg Wilkins <gregw@webtide.com> CTO http://webtide.com