Re: Re: Multi-path QUIC Extension Experiments

"Ma, Yunfei" <yunfei.ma@alibaba-inc.com> Thu, 15 July 2021 02:17 UTC

Return-Path: <yunfei.ma@alibaba-inc.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2393C3A1643 for <quic@ietfa.amsl.com>; Wed, 14 Jul 2021 19:17:45 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.096
X-Spam-Level:
X-Spam-Status: No, score=-2.096 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_EXCESS_BASE64=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=alibaba-inc.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fGYfqJRy0LOl for <quic@ietfa.amsl.com>; Wed, 14 Jul 2021 19:17:39 -0700 (PDT)
Received: from out0-145.mail.aliyun.com (out0-145.mail.aliyun.com [140.205.0.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EEC403A1640 for <quic@ietf.org>; Wed, 14 Jul 2021 19:17:38 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alibaba-inc.com; s=default; t=1626315455; h=Date:From:To:Message-ID:Subject:MIME-Version:Content-Type; bh=AXT7TvrobVY7BloyBnSy9zzH7/w3QGmFUWz5ua76vJw=; b=QynXrFcdrdzBAMW2VRZpb41bWImzO8qUGHQEMtvj4jAcKRlb2YzJXjxVIl/nP8RIdwq/OfA+TkUbEnoMd5OqYnNbpkAjwKf8sgHWki/hUzE0Fh8BZzRZ7W1YeB1ExgKLgFD8hpTqbMdxThFH2A974EQo7wnakEWxi6dkOImiCQc=
X-Alimail-AntiSpam: AC=PASS; BC=-1|-1; BR=01201311R731e4; CH=green; DM=||false|; DS=||; FP=0|-1|-1|-1|0|-1|-1|-1; HT=ay29a033018047193; MF=yunfei.ma@alibaba-inc.com; NM=1; PH=DW; RN=8; SR=0; TI=alimail_yun_mac.COREAPIb51f23f851b0464dadf68b8317cb9df1;
Received: from WS-web (yunfei.ma@alibaba-inc.com[alimail_yun_mac.COREAPIb51f23f851b0464dadf68b8317cb9df1]) by ay29a011140100061.et135 at Thu, 15 Jul 2021 10:17:34 +0800
Date: Thu, 15 Jul 2021 10:17:34 +0800
From: "Ma, Yunfei" <yunfei.ma@alibaba-inc.com>
To: Robin MARX <robin.marx@uhasselt.be>, Yanmei Liu <miaoji.lym@alibaba-inc.com>
Cc: quic <quic@ietf.org>, Christian Huitema <huitema@huitema.net>, Qing An <anqing.aq@alibaba-inc.com>, 李振宇 <zyli@ict.ac.cn>, "matt.joras" <matt.joras@gmail.com>, "lucaspardue.24.7" <lucaspardue.24.7@gmail.com>
Message-ID: <c8bd0e50-33f8-496a-ad48-ff4a06e679cf.yunfei.ma@alibaba-inc.com>
Subject: Re: Re: Multi-path QUIC Extension Experiments
X-Priority: 3
X-Mailer: [Alimail-Mailagent revision 2546561]
MIME-Version: 1.0
In-Reply-To: <CAC7UV9bXbVSy=RsBVecOw0feVGedUxo=wvHEFP-g+LDVOHDuGg@mail.gmail.com>
References: <02fa8c41-4673-47b6-b245-19ab13cc3fad.miaoji.lym@alibaba-inc.com>, <CAC7UV9bXbVSy=RsBVecOw0feVGedUxo=wvHEFP-g+LDVOHDuGg@mail.gmail.com>
x-aliyun-comment: CT_47
Content-Type: multipart/alternative; boundary="----=ALIBOUNDARY_94835_7fb3943a6700_60ef9abe_1bc2252"
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/cGgBPyaocWM5iAJyJyjgiSwNQ5w>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 15 Jul 2021 02:17:45 -0000

Hi Robin,

Thanks so much for your questions!

First, the head of line blocking discussed here is called multi-path head-of-line blocking or MP-HoL blocking, and its root cause is quite different from the stream HoL blocking usually discussed in QUICv1. The MP-HoL blocking happens when one path blocks the other path, not when one stream blocks the other stream. Please note that we indeed use multiple streams, for example, different video requests are carried in different QUIC streams. QUIC’s stream multiplexing ability and its benefits still hold in this scenario.

Second, regarding packet scheduling mode, right now, in our Taobao A/B test, we transmit packets on multiple paths simultaneously. However, you can definitely use traffic switching only and choose to switch when one path could not meet your bandwidth requirement. Basically, if you use multiple paths simultaneously, you get the most elasticity from a resource pooling perspective. It really comes down on what your application needs. We will also update the packet scheduling section soon in a newer version of the draft, in which we plan to include more discussions on the packet scheduling policy.

Third, regarding the benefits of more bandwith versus the "downsides". Whether you want more bandwidth depends on your application. For videos, yes, more bandwidth is extremely helpful in improving the long tail QoE, which is an important target for Taobao. We find multi-path QUIC helps us improve two important metrics, rebuffer rate and video start-up delays. In the past, if you work on multi-path scheduling that does not collaborate close enough with applications such as MPTCP, the MP-HoL blocking becomes the downside that cripples the performance. However, the user space nature of QUIC provides us the opportunity to solve this problem, so now our conclusion is that you can enjoy the benefits of more bandwidth and more reliable connectivity from multi-path without much of the “downsides”.

I hope my answer is helpful, but feel free to let me know if you have any additional comments.

Cheers,
Yunfei

from Alimail macOS
 ------------------Original Mail ------------------
Sender:Robin MARX <robin.marx@uhasselt.be>
Send Date:Wed Jul 14 07:39:37 2021
Recipients:Yanmei Liu <miaoji.lym@alibaba-inc.com>
CC:quic <quic@ietf.org>, Ma, Yunfei <yunfei.ma@alibaba-inc.com>, Christian Huitema <huitema@huitema.net>, Qing An <anqing.aq@alibaba-inc.com>, 李振宇 <zyli@ict.ac.cn>, matt.joras <matt.joras@gmail.com>, lucaspardue.24.7 <lucaspardue.24.7@gmail.com>
Subject:Re: Multi-path QUIC Extension Experiments

Hello Yanmei,

Thanks for the additional results on an interesting topic. I'm looking forward to reading the SIGCOMM paper.

I was a bit surprised to (apparently) see HOL blocking mentioned as a major issue, as that's one of the things QUIC aims to be better at than TCP.
It's a bit difficult to understand from the slides, but it seems like you're sending packets for a single stream (Stream ID 1 in the diagrams) on both the slow and fast path, which would indeed induce HOL blocking.
Consequently, I was wondering what the practical reasons are for you to multiplex packets for a single stream over multiple paths, as opposed to for example attaching a single stream to a single path (say: high priority streams use the fast path for all their packets).

I see this mentioned a bit in the draft under "packet scheduling", where it talks about switching paths once the cwnd is full for one. That indeed leads to the behaviour seen in the slides, but that's my question: why would you take those approaches then? 
Are there so many cases where the additional "bandwidth" from using multiple path's cwnd for a single stream outweigh the downsides of HOL blocking? Relatedly: what are the packet loss rates you've observed on real networks? 
Have you experimented with e.g., tying streams to paths more closely? Does that work better or worse? Why? 

I'm mainly wondering how these tradeoffs evolve depending on the type of paths available and if it's possible to make a model to drive this logic.
I assume there is much existing work on this for MPTCP, but I also assume some of that changes due to QUIC's independent streams / stream prioritization flexibility. 

Thank you in advance and with best regards,
Robin

On Sun, 11 Jul 2021 at 20:48, Yanmei Liu <miaoji.lym=40alibaba-inc.com@dmarc.ietf.org> wrote:

Hi everyone,

We have finished some experiments about deploying multi-path quic extension(https://datatracker.ietf.org/doc/draft-liu-multipath-quic/) in Alibaba Taobao short-form video streaming, and the experiment results are concluded in the slides (attached file).
If anyone is interested in the experimental details about multi-path quic, please let us know. 
All the feedbacks and suggestions are appreciated!

Best regards,
Yanmei

-- 


	dr. Robin Marx
	Postdoc researcher - Web protocols
	Expertise centre for Digital Media

	Cellphone +32(0)497 72 86 94 

	www.uhasselt.be
	Universiteit Hasselt - Campus Diepenbeek
	Agoralaan Gebouw D - B-3590 Diepenbeek
	Kantoor EDM-2.05