From nobody Thu Jan 6 16:24:57 2022 Return-Path: X-Original-To: bier@ietfa.amsl.com Delivered-To: bier@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4EE7A3A0B40; Thu, 6 Jan 2022 16:24:55 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.649 X-Spam-Level: X-Spam-Status: No, score=-1.649 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SdM5CS6BefJc; Thu, 6 Jan 2022 16:24:50 -0800 (PST) Received: from faui40.informatik.uni-erlangen.de (faui40.informatik.uni-erlangen.de [131.188.34.40]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D045B3A0B41; Thu, 6 Jan 2022 16:24:48 -0800 (PST) Received: from faui48e.informatik.uni-erlangen.de (faui48e.informatik.uni-erlangen.de [131.188.34.51]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by faui40.informatik.uni-erlangen.de (Postfix) with ESMTPS id A2954549F8E; Fri, 7 Jan 2022 01:24:40 +0100 (CET) Received: by faui48e.informatik.uni-erlangen.de (Postfix, from userid 10463) id 8BE554EA2B1; Fri, 7 Jan 2022 01:24:40 +0100 (CET) Date: Fri, 7 Jan 2022 01:24:40 +0100 From: Toerless Eckert To: Dirk Trossen Cc: Gyan Mishra , "bier-chairs@ietf.org" , BIER WG , Alvaro Retana Message-ID: References: <49abdeb209694f6cb4abae5a3ccfc23a@huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: <49abdeb209694f6cb4abae5a3ccfc23a@huawei.com> Archived-At: Subject: Re: [Bier] WGLC: draft-ietf-bier-multicast-http-response X-BeenThere: bier@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "\"Bit Indexed Explicit Replication discussion list\"" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2022 00:24:55 -0000 Happy new year everybody! Agree with Dirks points. Let me address three of my own: a) Personally i do like describing a solution and point out the BIER(-TE) specific design components. IMHO, if well done, an actual solution descript= ion is a lot more explanatory to operators than a purely architectural or proto= col spec document. Whether or not we have achieved the "well done" is of course a different qu= estion. b) Process wise, BIER-WG has adopted the document with the scope it had whe= n adopted. If we want to change the scope, this is potentially a different process iss= ue than just improving on it in its current scope. I have seen WG chairs easily approve without further adoption call split-up of documents into multiple p= arts, but i am not sure about the IETF BCP when changing scope fundamentally. c) Technically, i think there is a very strong tie between the solution/arc= hitecture description and BIER as the delivery mechanism:=20 If you wanted to do the same with IP Multicast, you have three options: I) For a stream offered in M different bitrates to N receivers, each of which could potentially want to receive any single one of those M bitrates at any point in time based on the path congestion towards that receiver, you statically create M*2^N ASM Multicast groups or SSM channels. Given how there may be K different media streaming simultaneous (K maybe short-tail of TV stations), it is easy to see how K*M*2^N easily exceeds any reasonable amount of hardware supported ASM/SSM multicast state in routers/switches for anything but an uninteresting small number of receivers (3 ? ;-). II) Instead of setting up those multicast trees statically, you let receivers dynamically join/leave an appropriate multicast group/channel when needed. This reduces the amount of group/channels to K*M, so its easy to see how this can scale to arbitrary amount of receivers. Alas, the amount of join/prune processing in the control plane (and pushing this into the forwarding plane) is extremely unlikely to be supportable in any router/switch not especially optimized in performance for this type of operation. Such as at least processing IGMP/MLS join/leave in distributed linecard CPU if not directly in FPE code. I have seen such prototypes for this in the past, but i am not aware of = production code for this. I would be surprised to even find guaranteed join/leave p= er second performance numbers on any router/switch product. And todays IP Multicast IPTV deployments typically have sender-caches that also provide channel change acceleration whereby IP Multicast is joined to as an optimization while still receiving a unicast stream after channel change. Which means that in todays deployment actual join latency is very uncritical, making lazy/bad router/switch implementations a non-issue. And hence moving to a solution where such channel change would have to be fast and precise e= ven more of a pain to expect from vendors. III) You can try to wiggle yourself through with a hybrid solution trying to mix&match I) and II) and assuming a lot more knowledge about about K, M and N, network topology and platform performances. Good luck! Probably could be a nice Master or PhD thesis, but not more. Long story short: It won't work in practice, and given the BIER alternat= ive, it would IMHO be a colossal waste of development resources (other than a happy student in need of a paper idea ;-). This is in the end what i tried to see contributed to the document. I am happy to improve on that part because it does IMHO give the core promotion for BIER which is what i think we want. And of course if we feel this type of core BIER promotion might be better perceived if we fork this part out of the current document, and the WG chairs would support this as= =20 a split-off WG document, i could try to find the time to propose such text. Wouldn't be just a few pages anyhow. And could serve (one of) the "archite= ctural" aspects Dirk i think was referring to. Cheers Toerless On Mon, Jan 03, 2022 at 08:19:28AM +0000, Dirk Trossen wrote: > Hi Gyan, >=20 > Many thanks for the comments (and a Happy New Year to you and the rest of= the list). >=20 > The summary of the key point you make is clear: the draft does not outlin= e an architecture but a specific solution. The focus on the latter makes it= hard to recognize the underlying architectural concept that differs from I= P multicast. In recent work, I called this =E2=80=98forward path requests, = return path multicast=E2=80=9D, making the implicit join operation for an i= nstantaneous multicast return operation clearer (I think). The draft, unfor= tunately (largely due to historical reasons), does not refer to this. >=20 > One direction of moving this work forward could be on that architectural = concept (with an HTTP-based solution being an example only). But it also be= gs the question whether BIER is the only realizing transport technology for= this concept? >=20 > Best, >=20 > Dirk >=20 > From: BIER [mailto:bier-bounces@ietf.org] On Behalf Of Gyan Mishra > Sent: 24 December 2021 08:24 > To: Dirk Trossen > Cc: bier-chairs@ietf.org; BIER WG ; Toerless Eckert ; Alvaro Retana > Subject: Re: [Bier] WGLC: draft-ietf-bier-multicast-http-response >=20 > Hi Dirk >=20 > Responses in-line >=20 > Kind Regards >=20 > Gyan >=20 > On Thu, Dec 23, 2021 at 4:24 AM Dirk Trossen > wrote: > Hi Gyan, >=20 > Many thanks for supporting this draft and its continuation as well as goo= d to see your interest in collaborate on the continued draft. > Gyan> Welcome! > See more inline. >=20 > Best, >=20 > Dirk >=20 > From: BIER [mailto:bier-bounces@ietf.org] O= n Behalf Of Gyan Mishra > Sent: 23 December 2021 00:15 > To: Toerless Eckert > > Cc: bier-chairs@ietf.org; BIER WG >; Alvaro Retana > > Subject: Re: [Bier] WGLC: draft-ietf-bier-multicast-http-response >=20 >=20 > Dear BIER WG, >=20 > I support publication of this informational draft which provides the fra= mework and basis of how stateless BIER infrastructure can eliminate state a= nd bandwidth usage with an aggregate http response when many client join a = stream. I would be happy to collaborate on the draft. > [DOT] Key here is that, unlike in IP multicast, there is no direct notion= of a stream but a joint interest (expressed in the request for a specific = chunk, for instance), expressed as individual unicast requests but replied = to as a single multicast response (there may be the same interest albeit ex= press some time later, resulting in a the different response albeit with li= kely same content). This is key since it makes the multicast nature more in= stantaneous, not following a longer-lived group-like semantic. Take an exam= ple of clients pulling video chunks. The lack of synchronization in the uni= cast requests for chunks leads inevitably (due to processing and packet lat= encies) to changing batches of requests that may be responded to with relev= ant content, although the content itself may be the same across several res= ponses. The batches likely change in membership since clients may encounter= different relevant latencies etc. > Gyan> You say =E2=80=9Cunlike IP Multicast=E2=80=9Dso what you are desc= ribing and optimizations is not a multicast solution but a completely new s= olution alternative for subscribers and operators. Is there a document I c= an read that explains how the host signaling works as it does not use IGMP = or PIM it seems and different process, procedures, FSM of how to express in= terest for a specific chunk and how that all works. That would really help= the reader in understanding the full scope of the use case. Also what par= t of IP Multicast is used as far as host signaling if any? > [DOT] This aspect is important in several technical aspects, such as timi= ng used to create the multicast response (therefore combining a certain num= ber of incoming requests), how to handle the congestion on multicast relati= ons that possible differ in nature from one response to another (since poss= ibly different clients are =E2=80=98combined=E2=80=99 into the individual r= esponses). To me, those questions need dealing with although not in this dr= aft IMO. What the draft provides is a base capability, as you also outline = it, to allow for this aggregation and the resulting multicast delivery of a= ggregated responses. >=20 > Gyan> Understood. I am still a bit lost from your above paragraph t= hat you state =E2=80=9Cunlike IP multicast=E2=80=9D > This sounds like a completely new architecture that does not use the mult= icast group concept for subscribers, but new framework to cutdown on IGMP a= nd PIM chatter, > I have some comments on the draft based on my experience working with mul= ticast video delivery: >=20 > Is the waste in transmission bandwidth due to PIM Assert and double strea= m when multiple first hop routers are connected to the source? >=20 > Can you comment on the increase in signaling when there are a few subscri= bers? I assume you are referring to IGMP join last hop router chatter. > [DOT] When comparing with IP multicast, its explicit group membership con= stitutes waste compared to the proposed mechanism since in the latter, the = =E2=80=98group=E2=80=99 is defined through batching requests (for the same = semantic content) at the server. The request for the content is here the = =E2=80=98signaling=E2=80=99 to compare against. Of course, for static group= s, you can argue that the joining happens once and the sender can simply st= ream, while the proposed mechanism signals for every chunk, so to speak. Bu= t the objectives are, of course, different since the underlying chunk retri= eval aims at a different semantic, not limited to a live stream. If you wan= t to move to such =E2=80=98individual viewing experiences=E2=80=99, the pro= posed solution still yields in multicast delivery due to the mere statistic= al synchronicity of chunk requests (particular for the short tail of a cata= logue). Also, if you do live event but provision them over an HTTP-based pl= atform, you are simply pushed into the unicast request/reply pattern =E2=80= =93 this draft proposes a mechanism that is based on the pattern in the for= ward direction but not the return one. > Gyan> As I stated above and I think should be included in the draft is a= detailed explanation of the proposed multicast replacement architecture ca= lled =E2=80=9Chttp level multicast=E2=80=9D and how it works detailed proc= edures and state machine. With HLS and DASH the video file is broken up in= to chunks segments so if you have to signal interest in the steam for evey = chunk that would be a lot of control plane signaling overhead where with IG= MP you join the group which is for the video feed not the individual chunks= of the video file being streamed. Am I missing something? > The goal of BIER is to eliminate state in the core for MVPN P-tree and C-= Tree tree building technologies and not the last hop router as that is on t= he customer side signaling. >=20 > I read through this RFC related to IRTF ICN deployments scenario discusse= d related to 80 nodes test, however not much detail is provided related to = the adaptive multicast streaming testing. >=20 > https://datatracker.ietf.org/doc/html/rfc8763 > [DOT] Right, RFC8763 mentions this scenario but indeed lacks details. The= test here was done as part of past efforts I led. The content was not live= but stored content. > Gyan> Understood. I think overall details of http level multicast is la= cking in the dradt > I read through this draft section 3.10 and it also does not provide detai= ls into how multicast adaptive video delivery works without having a gatewa= y or server that does an ingest of the adaptive video to source the multica= st distribution stream over UDP transport. >=20 > https://datatracker.ietf.org/doc/html/draft-ietf-bier-use-cases-12#sectio= n-3.10 > [DOT] You seem to assume a live streaming type of scenario, sourced from = an ingest point (over UDP) and relayed via this mechanism? But the draft ou= tlines something more akin to an HLS/DASH model, i.e., clients individually= request chunks and the server replies albeit aggregating several same cont= ent responses into multicast ones =E2=80=93 see also below. > Gyan> Understood. Ok so I think I get it this is a completely new archi= tecture multicast paradigm shift > [DOT] the aspects below are indeed all valid but it=E2=80=99s this part w= here I would expect input and positions from the application (protocol) com= munity since the draft is merely meant to provide a base delivery mechanism= , not an application level solution. In my past work, we had some insights = into the aspects you outline below but it=E2=80=99s not captured in this dr= aft for the reasons already mentioned. > Gyan> Understood > The key point I am missing is how adaptive video is natively possible ove= r UDP transport. >=20 > Comments below are related to multicast and adaptive streaming. The prob= lem with multicast is it uses UDP transport and is a one way stream and not= bidirectional so no feedback mechanism exists for adaptive rate limiting. = I believe QUIC with sideband TCP channel is a possibility which I have in= vestigated as an alternate transport to UDP. PGM pragmatic multicast is an= other possibility. >=20 > https://datatracker.ietf.org/doc/html/rfc3208 >=20 >=20 > My understanding and experience with HLS and DASH adaptive streaming whic= h uses chunks and segments natively both video delivery methods only suppor= t Unicast video delivery. Does not support native delivery via multicast. > [DOT] It is the HLS/DASH type of delivery mechanism. The lack of support = for multicast here motivated the draft initially but still leaves questions= open on issues like timing of aggregation, interaction with upper layer me= chanism, how to handle congestion control on the return path (if at all)=E2= =80=A6 > Gyan> Understood > [DOT] as an example: DASH adaptive streaming mimics TCP in the sense of u= sing latency information as an indication for network conditions, thereby l= owering the stream quality when such latency occurs. If you look into the i= dea of aggregated chunk responses (as in the draft here), you may notice qu= ickly that =E2=80=98delaying=E2=80=99 chunk responses is beneficial here si= nce doing so increases your window to =E2=80=98catch=E2=80=99 more incoming= client requests for the same content, thereby increasing the size of the r= esponse group. How long to delay? Well, could be something like 75% of the = chunk length (just to give a number) since it gives you a good delay budget= for returning the response and playing it out (with chunk length of 4 or m= ore seconds). Problem is that if you do that, DASH adaptive stream will kic= k in and reduce your streaming quality=E2=80=A6so there are interactions th= at need to be understood. > Gyan> Understood > Most all modern web browsers Chrome, Firefox and Safari etc do not suppor= t NPAPI plug in and thus cannot play multicast natively. That is a major i= ndustry wide problem. >=20 > There are standard solutions that exist that involve a UDP wrapper at the= source multicast gateway that ingests unicast and distributes multicast. >=20 > Ramp is one vendor that provides such a solution. > Ramp uses a source sender that ingests HLS or DASH unicast stream and pac= kages in UDP envelope and at the host receiver endpoint Window or Mac it st= rips UDP encapsulation and plays out HLS or DASH video natively on loopback= 127.0.0.1. >=20 > https://www.rampecdn.com/ >=20 > Wowza is another vendor that does support multicast streaming but require= s ingest as well of adaptive steam and then sources multicast distribution = stream. >=20 > https://www.wowza.com/community/t/multicast-options-with-wowza-streaming-= engine/44969 >=20 >=20 > There has been development work done with WebRTC messaging that has the a= bility to provide a similar solution to natively play out multicast in a we= b browser. >=20 > https://www.w3.org/2020/09/webrtc-charter.html >=20 > https://datatracker.ietf.org/wg/rtcweb/documents/ >=20 > This is the only link below I could find on DASH multicast. >=20 > https://res-www.zte.com.cn/mediares/magazine/publication/com_en/article/2= 01803/PARK.pdf >=20 > Some history on DVB: >=20 > DVB is basically MPEG video delivery over an IP network. There was a DVB= IETF WG which concluded decades ago and since had been a standards forum f= or digital video delivery. >=20 > https://datatracker.ietf.org/wg/ipdvb/documents/ >=20 >=20 > https://dvb.org/ >=20 > This document was published in 2018 and talks about adaptive streaming ov= er IP multicast. >=20 > This document talks about how ingest occurs if adaptive video source stre= am done by a multicast server which encapsulated and delivers the multicast= over the network. >=20 > https://dvb.org/wp-content/uploads/2019/12/a176_adaptive_media_streaming_= over_ip_multicast_2018-02-16_draft_bluebook.pdf >=20 > 3GPP 5G to support MBS Multicast and Broadcast Services with release 17 a= daptive video delivery MPEG DASH >=20 > https://www.smpte.org/webcast/an-introduction-to-3gpp-5g-multicast-and-br= oadcast-services?hsLang=3Den >=20 > https://dashif.org/docs/workshop-2019/12-Stockhammer%20Multicast%20Decemb= er%202019.pdf >=20 > https://www.etsi.org/deliver/etsi_ts/103700_103799/103720/01.01.01_60/ts_= 103720v010101p.pdf >=20 > This solution to use BIER could apply to any network that utilizes adapti= ve video on FBB, MBB as well as an other network so could have ubiquitous = use cases. > [DOT] Note again, that the point here is that the solution is based on a = request/response semantic with the former in unicast and the latter possibl= y in multicast (if more than one request to the same content). Most of the= solutions you outline above are joining an explicit group for a stream. No= w you could, of course, model this via an HTTP/2 request and a stream of HT= TP DATA frames. So this would make the forward request your explicit group = membership signaling. > Gyan> Understood. I am getting it now a completely new multicast archit= ecture > I can provide a further review of the draft and input to help advance the= document. > [DOT] Great, thanks! Your email was already extremely helpful. > Gyan> Most welcome!! > Many Thanks! >=20 > Gyan >=20 > On Mon, Dec 20, 2021 at 3:13 PM Toerless Eckert > wrote: > Dear BIER WG-chairs >=20 > a) As authors, we are pinging other WG members to ask for more reviews wi= thin BIER-WG. >=20 > b) Picking up on Alvaros suggestion (thanks a lot), i would like to reque= st > on behalf of the authors a TSVART (Transport Area Review Team). >=20 > (we will ask for Application Area review later.. working ourselves up the= stack). >=20 > Here is the suggested text for the datatracker form through which > you WG chairs could reques >=20 > Deadline: >=20 > 02-15-2022 >=20 > Document revision: >=20 > draft-ietf-bier-multicast-http-response-06 >=20 > Requester's comments and instructions >=20 > The authors are requesting early review of the document from transport = area. >=20 > The following is a motivational summary of the purpose of this document: >=20 > Adaptive bitrate streaming (such as today DASH) for live-streaming or s= hort-tail can > use Multicast to improve network efficiency by eliminating duplicate tr= ansmission > of the same data segments to multiple users requiring the same bitrate.= This > was shown and used in products as early as 200x (e.g.: Digital Fountain= ). > These solutions never proliferated because of the intrinsic signaling p= roblems > of IP Multicast causing severe performance/scale/operational issues. >=20 > These IP Multicast limitations can today be overcome with the novel BIER > multicast mechanism. This document describes an application solution la= yer > architecture in which BIER is used as the multicast mechanism for HTTP > adaptive bitrate streaming including the use of BIER-TE for including > path steering optimizations into the solution. This document is solely > informational and does not intend to establish any standards for the IE= TF. >=20 > Thank you so much > Toerless (for the authors) >=20 > On Fri, Dec 17, 2021 at 05:09:36AM -0800, Alvaro Retana wrote: > > Greg: > > > > Hi! > > > > It is important in light of rfc8789, which requires that all IETF docum= ents > > reach consensus, to demonstrate that the WG cares and has reviewed, > > commented, etc. Silence is not a good indicator of consensus. Thanks f= or > > following up on this! > > > > If there is interest in the WG, and given that this document deals with > > HTTP level multicast, please request an early review from both the ART = and > > Transport areas review teams before sending it off for publication. > > > > > > A couple of quick comments for the authors. Please use the rfc8174 > > template (not rfc2119). I am surprised that there are no Normative > > references ( > > https://www.ietf.org/about/groups/iesg/statements/normative-informative= -references/ > > ). > > > > Thanks! > > > > Alvaro. > > > > On December 9, 2021 at 12:19:05 PM, Greg Shepherd (gjshep@gmail.com) wrote: > > > > Zero response for this WGLC, not even from the authors. Is this work st= ill > > of interest to anyone in the WG? > > > > - Shep > > > > On Mon, Nov 22, 2021 at 8:45 AM Greg Shepherd > wrote: > > > > > This email starts a 2 week WGLC timer for > > > draft-ietf-bier-multicast-http-response. Please read and review: > > > > > > https://datatracker.ietf.org/doc/draft-ietf-bier-multicast-http-respo= nse/ > > > > > > We also need a doc Shepherd before we can progress this work. Can som= eone > > > please step up to help. > > > > > > Thanks, > > > - Shep > > > (Chairs) > > > > > _______________________________________________ > > BIER mailing list > > BIER@ietf.org > > https://www.ietf.org/mailman/listinfo/bier >=20 > > _______________________________________________ > > BIER mailing list > > BIER@ietf.org > > https://www.ietf.org/mailman/listinfo/bier >=20 >=20 > -- > --- > tte@cs.fau.de >=20 > _______________________________________________ > BIER mailing list > BIER@ietf.org > https://www.ietf.org/mailman/listinfo/bier > -- >=20 > [http://ss7.vzw.com/is/image/VerizonWireless/vz-logo-email] >=20 > Gyan Mishra >=20 > Network Solutions Architect >=20 > Email gyan.s.mishra@verizon.com >=20 > M 301 502-1347 >=20 > -- >=20 > [http://ss7.vzw.com/is/image/VerizonWireless/vz-logo-email] >=20 > Gyan Mishra >=20 > Network Solutions Architect >=20 > Email gyan.s.mishra@verizon.com >=20 > M 301 502-1347 >=20 --=20 --- tte@cs.fau.de