Re: More details about multipath quic in Alibaba

David Schinazi <dschinazi.ietf@gmail.com> Sun, 25 October 2020 19:34 UTC

Return-Path: <dschinazi.ietf@gmail.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9C85A3A158E for <quic@ietfa.amsl.com>; Sun, 25 Oct 2020 12:34:54 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Level:
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vlIIvqPm46wq for <quic@ietfa.amsl.com>; Sun, 25 Oct 2020 12:34:52 -0700 (PDT)
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com [IPv6:2a00:1450:4864:20::12d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6FDD43A158C for <quic@ietf.org>; Sun, 25 Oct 2020 12:34:52 -0700 (PDT)
Received: by mail-lf1-x12d.google.com with SMTP id 77so9153392lfl.2 for <quic@ietf.org>; Sun, 25 Oct 2020 12:34:52 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Wrf3sZUNJEikwz18DCK+uuq8WLP1x05xEY7z22cOjds=; b=jNLaSY2kpp5Inoe0cOXg3zsciwQe8OwKoRt1gKODOAIaITjxyxgDY8/aGU/b8CTbWC AkxaMLD0C7k0ysOHcm6BfFhp40RzTqdIiJbIHKsEcsTQrEfIOqPqvlytECVCLeaWupP9 6Rtjj4FDcc3xv5HhMWDWrTULBTmdNnPQckIQiDa6/859hk4e30FS5BE8XKE/krhQgV3G nF5MDCVQRYawDZzb9ontA+V0geiFqwxAkDzbs4Z4N7BoMZubcrJGLek9TaDK/ngjIbU7 w8GEa86pbxqshZnfntSzG0yfhImMoRN0KGZq7wKA44WsQ9pa0bsEdhRt+CVw7xpm5Xes qH+g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Wrf3sZUNJEikwz18DCK+uuq8WLP1x05xEY7z22cOjds=; b=a6Fh5jWAD4PzU0iiP+T//yV+OW0jUNPXLyzYvb2ON+zc4hi51l0ULjsdpCx3XGkIu5 r+F6uM624hRfmL2x21CoPGyHGffIYwWvzq6vO1mM6CeoajpUnkaEVEIC9yw6O/AfeE2h pQ6o0+9DZBQ/EMokWniaATFHrpAjWKfOimqqLJmmpvu4CLRlxPFBIg0liWIGtvSsJUlK 8EiExh3rGILjT3RLCsoyJ668PuAXTXioZKfGm7lvbq9jcBuvclv6M/YcZwXtxr2/oKaE u7ULMkNpg7p5p6MiJ0NW0aAZZ6JhLzOrQLqGKl95/PHhD7EaypjlSx41I1BktyDWBXmo Kz+g==
X-Gm-Message-State: AOAM533SP3lDw4TkkrmvBiht50JGwTwNxOifavKGQ+1pkdkwgY2swHb9 8mfcOkmBw+rXK2YQeCwHrjjbyjNRF9HxfVoBLs59Tbj2n40=
X-Google-Smtp-Source: ABdhPJxEvrcZqzjggbsGB29cbweoroV/QueMWsAmBWxHCUHDHidWeQSrWyFC7FeHv0h9fOCH3kMPivawLvttvTw3gOU=
X-Received: by 2002:a19:64e:: with SMTP id 75mr4320851lfg.143.1603654490669; Sun, 25 Oct 2020 12:34:50 -0700 (PDT)
MIME-Version: 1.0
References: <CAMDWRAb5pWdYHwc5h6MSc6uprtUQ+bWniMWamiv+mVnX32yQXA@mail.gmail.com>
In-Reply-To: <CAMDWRAb5pWdYHwc5h6MSc6uprtUQ+bWniMWamiv+mVnX32yQXA@mail.gmail.com>
From: David Schinazi <dschinazi.ietf@gmail.com>
Date: Sun, 25 Oct 2020 12:34:39 -0700
Message-ID: <CAPDSy+7Tx-wSG6+z97XdYGt6zxQfU_Lfp7axHjfz3xrAR43iDg@mail.gmail.com>
Subject: Re: More details about multipath quic in Alibaba
To: Yanmei Liu <healing4d@gmail.com>
Cc: QUIC <quic@ietf.org>, miaoji.lym@alibaba-inc.com, yunfei.ma@alibaba-inc.com, anqing.aq@alibaba-inc.com, hongqiang.liu@alibaba-inc.com
Content-Type: multipart/alternative; boundary="00000000000042480205b283e8db"
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/1jFiKORR6PaxgFhF582SSVVOfhg>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 25 Oct 2020 19:34:55 -0000

Thanks for these details Yanmei!

A few followup questions inline:

On Sat, Oct 24, 2020 at 11:45 PM Yanmei Liu <healing4d@gmail.com> wrote:

> Hi everyone,
>
> After we introduced the use cases and requirements of Multipath QUIC
> project at Alibaba in the IETF Interim meeting on Oct. 22rd., we have
> received many questions from people who are curious about the details in
> the working group, so we would like to give more information in this
> mailing list.
>
> - Q1: Do we care about cellular cost?
> - Sure. Cellular cost is a very important factor that influences our
> packet scheduler design. Some of our customers are sensitive about the
> cellular cost and some do not, so they need different scheduling policies.
> For customers who are sensitive about cellular cost, Alibaba have
> collaborated with several mobile carriers, so customers can get enrolled in
> a special data plan, which allows them to enjoy unlimited free cellular
> data when using apps from Alibaba eco-system(i.e. alibaba traffic costs the
> user nothing). For customers who are not sensitive about the data cost, for
> example, professional streamers/Vloggers and customers who want to get high
> quality services for business collaborations, they can turn on the
> multi-path feature if they want.
>

That's really cool. Out of curiosity, how does your application know that
the user has unlimited free cellular data or not?


> - Q2: If someone who wants this could go off an cook up an extension and
> get experience with it then come back with that experience?
> - We completely agree that a large-scale deployment experience is needed
> to draw a conclusion and inspire new innovations. Right now, we are doing
> the extension described in
> https://tools.ietf.org/html/draft-an-multipath-quic-00, which is
> integrated in Taobao for short-form video streaming. The feature is already
> made available for certain customers in China and we are working to expand
> the scope.
>

This is great! If by any chance you have the time to run an experiment
between connection migration and full multipath that would be incredibly
valuable.


> - Q3: How to solve the linkability problem in multipath ?
> - QUICv1 solves the linkablity problem with Connection ID’s re-negotiation
> when connection migration happens. We reuse the mechanism from QUICv1 in
> our multipath quic implementation. For each new sub-connection, client and
> server need to exchange available unused Connection IDs before a client
> tries to create a new sub-connnection. When client creates a new
> sub-connection and sends packets on the new path, it uses the new
> Connection IDs. As the process of the negotiation is encrypted, it would
> not link the packets sent on new path with the previous path.
>

That's a great plan.

- Q4: Difference from early multipath QUIC proposals?
> - First of all, we really appreciate the work of early proposals such as
> [I-D.deconinck-quic-multipath]. However, in our implementation, we’ve found
> that two things are extremely important and should impact the protocol
> design. First, we build multi-paths based on the concept bi-directional
> “sub-connections” in each new path, and find it’s the easiest way to
> implement multipath because it readily fits into the nature of both
> cellular and wifi links that cover the majority of multi-path applications.
> In doing so, you can almost reuse all the design / implementations for
> QUICv1 connections. Second,  the multi-path QUIC design needs dynamic
> scheduling strategy. As is pointed in the answer to Q1 above, our customers
> who want multipath have very different needs, so making the scheduler
> dynamic allows us to fulfill the job.
>
> We would like to know what people think about the draft, and all feedbacks
> and suggestions are welcome.
>

I'd have to implement this to form a full opinion, but I think I like this
design - placing the complexity on top of QUIC instead of inside QUIC
sounds like a great way to simplify implementations.

David

Thanks,
> Yanmei Liu
>