Re: Multi-path QUIC Extension Experiments
Charles 'Buck' Krasic <charles.krasic@gmail.com> Fri, 16 July 2021 19:50 UTC
Return-Path: <charles.krasic@gmail.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D5EE53A4269 for <quic@ietfa.amsl.com>; Fri, 16 Jul 2021 12:50:38 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Level:
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aPEWV9OPAlZE for <quic@ietfa.amsl.com>; Fri, 16 Jul 2021 12:50:33 -0700 (PDT)
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com [IPv6:2a00:1450:4864:20::12b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9866E3A4262 for <quic@ietf.org>; Fri, 16 Jul 2021 12:50:32 -0700 (PDT)
Received: by mail-lf1-x12b.google.com with SMTP id i5so17872087lfe.2 for <quic@ietf.org>; Fri, 16 Jul 2021 12:50:32 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=HhrcbYbgi79Q6pe7+11ESMd+v9XGl3EvmniYx3AvdfA=; b=SjJHCYh+XIveI480iSfSLYM/Sn0WV5ZAXkLEnmyrhLlBy0MD0xp5eu9qcjysQwm6zp 1A0VXA02RikA98DYyJ5vRll6dKmpvtNMZUiCWEaMzcVNq75Y8SjSbdedgIKCkXjoXmZA 96Us2M4bn5OXm2XwYQ4qN5Ws2th9iCn2Yy202AXpgT+ZjlOSCy4uucmfLR1SbNklbSVS EF3y9yTG7s53EsQc7ywNBczjC4qj3/l5f2qXGTpvlHwuGkjfilf/8eZILE3L9w7VSo8B 8tZwVaDA18Ds8pErQm+Wa/reuH5cnF68d5uIlUqQ2fs4S/rMZnKLvlU8fEMj65absv6Y sYeQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=HhrcbYbgi79Q6pe7+11ESMd+v9XGl3EvmniYx3AvdfA=; b=qCcPV6wEz52XrpyqB5EdVdqQx6dC2DeqF+ow4HY7SR4jQGoWzA3Tq9TNMJ+mAYokI7 3Wx6uvG4tDyeLckh9h7JClEYi73S62zoLdbGx00cRd8mQOQb/QeOpXPWBl6lhvdx7u1q sGsZnw8Raii+Q+pBnZ4OL0GfvZy4HV7i0vM/Uk8IQ7wRNsqdk7AYCqrXztEbKtOl3lPa VXBLbQhW24P9V0YR0zKrwcNnZT81dp9vAoeWNBEjilCKNdBMlGvWFnLQMfSy4rYa9jFv o98zD1W3dYQNaEMLzd1buSlVtFuLHBy2cZq7lGzR3Ldx+8OlZanzeuKfpEatbA0pvkWF lHUA==
X-Gm-Message-State: AOAM533h/G+KkrMsukxZmMrVAs+umZM1WZeoQhpX2UYrq/YhOso8G56Y maBJwG9TfbX/6a99ljSU187cuGx/I4WAfsGSrBM=
X-Google-Smtp-Source: ABdhPJwGSv8BFSopmAEefanAHJ0ygTJAnFk8aukaPnm+E/hbjSS/Gmgggk9Z8G3Fl780FgcG2W6nKtrGgIeln4ZvtHM=
X-Received: by 2002:a19:5053:: with SMTP id z19mr9171072lfj.498.1626465030123; Fri, 16 Jul 2021 12:50:30 -0700 (PDT)
MIME-Version: 1.0
References: <8C2E8EFB-756B-449B-84E0-11CD6B57E541@ericsson.com> <0334A48E-B6C6-464C-A48C-4512A453DA81@fb.com>
In-Reply-To: <0334A48E-B6C6-464C-A48C-4512A453DA81@fb.com>
From: Charles 'Buck' Krasic <charles.krasic@gmail.com>
Date: Fri, 16 Jul 2021 12:50:18 -0700
Message-ID: <CAPhuoz0vz2k63_ZaWmUg_XgSHUopid7vf+JY=JVFm_VqQJY87w@mail.gmail.com>
Subject: Re: Multi-path QUIC Extension Experiments
To: Roberto Peon <fenix=40fb.com@dmarc.ietf.org>
Cc: Mirja Kuehlewind <mirja.kuehlewind=40ericsson.com@dmarc.ietf.org>, "Ma, Yunfei" <yunfei.ma=40alibaba-inc.com@dmarc.ietf.org>, Robin MARX <robin.marx@uhasselt.be>, Yanmei Liu <miaoji.lym@alibaba-inc.com>, "matt.joras" <matt.joras@gmail.com>, 李振宇 <zyli@ict.ac.cn>, Christian Huitema <huitema@huitema.net>, "lucaspardue.24.7" <lucaspardue.24.7@gmail.com>, quic <quic@ietf.org>, Qing An <anqing.aq@alibaba-inc.com>
Content-Type: multipart/alternative; boundary="0000000000005c350905c742e691"
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/AI3g9Ph5QuFVvzN5jSHM11L1x78>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 16 Jul 2021 19:50:39 -0000
"don't overcommit" includes the common practice of setting very large limits on the client side, where in aggregate the case of server being flow control limited is effectively non-existent. I am curious to hear clarification of the precise definition of MP-HoL blocking here. is it not flow control, but rather path aliasing where distinct paths are actually sharing some physical link(s)? On Fri, Jul 16, 2021 at 12:13 PM Roberto Peon <fenix=40fb.com@dmarc.ietf.org> wrote: > I too am curious! > There are only two ways to handle flow control—overcommit, or don’t > overcommit. > > The “don’t overcommit” choice leads to blocking, since any of that > resource allocated to one path can’t be used by the other. > > The “overcommit” choice either leads to OOM, or throwing out some > successfully transmitted and received data. > > > Underlying this is a fun question: Which inefficiency is worse? Not using > resources that should be used (i.e. from choosing to not overcommit), or > sometimes redundantly using a resource (from choosing to overcommit)? > I’m curious too about what implementation strategies we end up doing in > general around this, and.. if enough implementations are choosing > overcommit, if we need some different protocol mechanisms to bound the > redundancy? > -=R > > > > *From: *QUIC <quic-bounces@ietf.org> on behalf of Mirja Kuehlewind > <mirja.kuehlewind=40ericsson.com@dmarc.ietf.org> > *Date: *Friday, July 16, 2021 at 6:15 AM > *To: *"Ma, Yunfei" <yunfei.ma=40alibaba-inc.com@dmarc.ietf.org>, Robin > MARX <robin.marx@uhasselt.be>, Yanmei Liu <miaoji.lym@alibaba-inc.com> > *Cc: *"matt.joras" <matt.joras@gmail.com>, 李振宇 <zyli@ict.ac.cn>, > Christian Huitema <huitema@huitema.net>, "lucaspardue.24.7" < > lucaspardue.24.7@gmail.com>, quic <quic@ietf.org>, Qing An < > anqing.aq@alibaba-inc.com> > *Subject: *Re: Multi-path QUIC Extension Experiments > > > > Hi Yunfei, > > > > thanks as well for you sharing your results! Can you explain even a bit > more what you mean by MP-HoL Blocking? Is this because of the flow control > limits? If so wouldn’t it make sense to reserve a certain “space” for each > path? > > > > Mirja > > > > > > *From: *QUIC <quic-bounces@ietf.org> on behalf of "Ma, Yunfei" <yunfei.ma= > 40alibaba-inc.com@dmarc.ietf.org> > *Date: *Thursday, 15. July 2021 at 04:18 > *To: *Robin MARX <robin.marx@uhasselt.be>, Yanmei Liu < > miaoji.lym@alibaba-inc.com> > *Cc: *"matt.joras" <matt.joras@gmail.com>, 李振宇 <zyli@ict.ac.cn>, > Christian Huitema <huitema@huitema.net>, "lucaspardue.24.7" < > lucaspardue.24.7@gmail.com>, quic <quic@ietf.org>, Qing An < > anqing.aq@alibaba-inc.com> > *Subject: *Re: Re: Multi-path QUIC Extension Experiments > > > > Hi Robin, > > > > Thanks so much for your questions! > > > > First, the head of line > blocking discussed here is called multi-path head-of-line blocking or MP-HoL blocking, and its root cause is quite different from the stream HoL blocking usually discussed in > QUICv1. The MP-HoL blocking happens when one path blocks the other path, not when one stream blocks the other stream. Please note that we indeed > use multiple streams, for example, different video requests are carried in different QUIC streams. QUIC’s stream multiplexing ability and its benefits still hold in this scenario. > > > > Second, regarding packet scheduling mode, > right now, in our Taobao A/B test, we transmit packets on multiple paths simultaneously. However, you can definitely use > traffic switching only and choose to switch when one path could not meet > your bandwidth requirement. Basically, if you use multiple paths > simultaneously, you get the most elasticity from a resource pooling > perspective. > It really comes down on what your application needs. We will also update the packet scheduling section > soon in a newer version of the > draft, in which we plan to include more discussions on the packet scheduling > policy. > > > > Third, regarding the benefits of more bandwith versus the "downsides". > Whether you want more bandwidth depends on your application. For videos, yes, more bandwidth is > extremely helpful in improving the long tail QoE, which is an important target for Taobao. We find multi-path QUIC helps us improve two important metrics, rebuffer rate and video start-up delays. > In the past, if you work on multi-path scheduling that does not collaborate > close enough with applications such as MPTCP, the MP-HoL blocking becomes > the downside that cripples the > performance. However, the user space nature of QUIC provides us the opportunity to solve this problem, > so now our conclusion is that > you can enjoy the benefits of more bandwidth and more reliable connectivity > from multi-path without much of the “downsides”. > > > > I hope my answer is helpful, but feel free to let me know if you have any > additional comments. > > > > Cheers, > > Yunfei > > > > from Alimail macOS > <https://protect2.fireeye.com/v1/url?k=7cc82aa7-2353138a-7cc86a3c-8692dc8284cb-e08a325a5c75cf95&q=1&e=de295b4f-9105-4e32-980f-779c711eaa62&u=https://mail.alibaba-inc.com/> > > ------------------Original Mail ------------------ > > *Sender:*Robin MARX <robin.marx@uhasselt.be> > > *Send Date:*Wed Jul 14 07:39:37 2021 > > *Recipients:*Yanmei Liu <miaoji.lym@alibaba-inc.com> > > *CC:*quic <quic@ietf.org>, Ma, Yunfei <yunfei.ma@alibaba-inc.com>, > Christian Huitema <huitema@huitema.net>, Qing An < > anqing.aq@alibaba-inc.com>, 李振宇 <zyli@ict.ac.cn>, matt.joras < > matt.joras@gmail.com>, lucaspardue.24.7 <lucaspardue.24.7@gmail.com> > > *Subject:*Re: Multi-path QUIC Extension Experiments > > Hello Yanmei, > > > > Thanks for the additional results on an interesting topic. I'm looking > forward to reading the SIGCOMM paper. > > > > I was a bit surprised to (apparently) see HOL blocking mentioned as a > major issue, as that's one of the things QUIC aims to be better at than TCP. > > It's a bit difficult to understand from the slides, but it seems like > you're sending packets for a single stream (Stream ID 1 in the diagrams) on > both the slow and fast path, which would indeed induce HOL blocking. > > Consequently, I was wondering what the practical reasons are for you to > multiplex packets for a single stream over multiple paths, as opposed to > for example attaching a single stream to a single path (say: high priority > streams use the fast path for all their packets). > > > > I see this mentioned a bit in the draft under "packet scheduling", where > it talks about switching paths once the cwnd is full for one. That indeed > leads to the behaviour seen in the slides, but that's my question: why > would you take those approaches then? > > Are there so many cases where the additional "bandwidth" from using > multiple path's cwnd for a single stream outweigh the downsides of HOL > blocking? Relatedly: what are the packet loss rates you've observed on > real networks? > > Have you experimented with e.g., tying streams to paths more closely? Does > that work better or worse? Why? > > > > I'm mainly wondering how these tradeoffs evolve depending on the type of > paths available and if it's possible to make a model to drive this logic. > > I assume there is much existing work on this for MPTCP, but I also assume > some of that changes due to QUIC's independent streams / stream > prioritization flexibility. > > > > Thank you in advance and with best regards, > > Robin > > > > > > On Sun, 11 Jul 2021 at 20:48, Yanmei Liu <miaoji.lym= > 40alibaba-inc.com@dmarc.ietf.org> wrote: > > Hi everyone, > > We have finished some experiments about deploying multi-path quic > extension(https://datatracker.ietf.org/doc/draft-liu-multipath-quic/) in > Alibaba Taobao short-form video streaming, and the experiment results are > concluded in the slides (attached file). > If anyone is interested in the experimental details about multi-path quic, > please let us know. > All the feedbacks and suggestions are appreciated! > > Best regards, > Yanmei > > > > > -- > > > > *dr. Robin Marx* > > Postdoc researcher - Web protocols > > Expertise centre for Digital Media > > > > *Cellphone *+32(0)497 72 86 94 > > > > www.uhasselt.be > <https://protect2.fireeye.com/v1/url?k=37557dd4-68ce44f9-37553d4f-8692dc8284cb-fe608437d16ed9d9&q=1&e=de295b4f-9105-4e32-980f-779c711eaa62&u=http://www.uhasselt.be/> > > Universiteit Hasselt - Campus Diepenbeek > > Agoralaan Gebouw D - B-3590 Diepenbeek > > Kantoor EDM-2.05 > > > > [image: Image removed by sender.] > > > >
- Multi-path QUIC Extension Experiments Yanmei Liu
- Re: Multi-path QUIC Extension Experiments Robin MARX
- Re: Re: Multi-path QUIC Extension Experiments Ma, Yunfei
- Re: Multi-path QUIC Extension Experiments Mirja Kuehlewind
- Re: Multi-path QUIC Extension Experiments Roberto Peon
- Re: Multi-path QUIC Extension Experiments Charles 'Buck' Krasic
- Re: Multi-path QUIC Extension Experiments Yunfei Ma
- Re: Multi-path QUIC Extension Experiments Mikkel Fahnøe Jørgensen
- Re: Multi-path QUIC Extension Experiments Roberto Peon
- Re: Multi-path QUIC Extension Experiments Christian Huitema
- Multi-path QUIC Extension Experiments Alexis Norech
- Re: Multi-path QUIC Extension Experiments Charles 'Buck' Krasic
- Re: Multi-path QUIC Extension Experiments Roberto Peon
- Re: Multi-path QUIC Extension Experiments Lucas Pardue
- Re: Multi-path QUIC Extension Experiments Yunfei Ma
- Re: Multi-path QUIC Extension Experiments Yunfei Ma
- Re: Multi-path QUIC Extension Experiments Yunfei Ma
- Re: Multi-path QUIC Extension Experiments Christian Huitema
- Re: Multi-path QUIC Extension Experiments Lucas Pardue
- Re: Multi-path QUIC Extension Experiments Lars Eggert
- Re: Multi-path QUIC Extension Experiments Robin MARX
- Re: Multi-path QUIC Extension Experiments Mikkel Fahnøe Jørgensen
- Re: Multi-path QUIC Extension Experiments Mikkel Fahnøe Jørgensen
- Re: Multi-path QUIC Extension Experiments Behcet Sarikaya
- Re: Multi-path QUIC Extension Experiments Christian Huitema
- Re: Multi-path QUIC Extension Experiments Yunfei Ma
- Re: Multi-path QUIC Extension Experiments Ian Swett
- Re: Multi-path QUIC Extension Experiments Yunfei Ma
- Re: Multi-path QUIC Extension Experiments Spencer Dawkins at IETF
- Re: Multi-path QUIC Extension Experiments Christian Huitema
- Re: Multi-path QUIC Extension Experiments Spencer Dawkins at IETF
- Re: Multi-path QUIC Extension Experiments Yunfei Ma