Re: How to avoid kernel's receive buff full , then UDP packet dropped by kernel?

Ian Swett <ianswett@google.com> Mon, 02 December 2019 17:33 UTC

Return-Path: <ianswett@google.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AE1981201EA for <quic@ietfa.amsl.com>; Mon, 2 Dec 2019 09:33:22 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -17.499
X-Spam-Level:
X-Spam-Status: No, score=-17.499 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qKsrGsqN1Q0M for <quic@ietfa.amsl.com>; Mon, 2 Dec 2019 09:33:20 -0800 (PST)
Received: from mail-ua1-x933.google.com (mail-ua1-x933.google.com [IPv6:2607:f8b0:4864:20::933]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 34C8212007A for <quic@ietf.org>; Mon, 2 Dec 2019 09:33:20 -0800 (PST)
Received: by mail-ua1-x933.google.com with SMTP id i31so159107uae.13 for <quic@ietf.org>; Mon, 02 Dec 2019 09:33:20 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=xlluV8OeSeLmI150vECIReGdlviJ5L2c4uPoEKfFcxs=; b=n2PZ63yaBfbQGCUhsUXiYgl2+WNcLwEWkJYNxYOzgtzJKc/loJW5kWD9Useik6Rsd7 WdgvcC2cvp9E3FtHPh0IWk/neZ6CTrOvIngJ6FK/TQ4Prlm8Pkbo41xcGvRT7mMbUyTr i5TOJEOh50ykbDHg14gdNbkXUqnlAHMmbNe9SJTuPS/7ZekUjk7E2kAbjVetoYqoCR7x uquJo11V0VdMPUlLwdxzE0P55H2GVteFLcoX12nD/UiboLxMVuxdWGriGBXOlit9N2UE erStS0DLRj6HjpiKZon6YA7iy65N3vcZt9RbV2jj0Be80FOQqmjdPxrUxgsTV4AVMVrv FEUg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=xlluV8OeSeLmI150vECIReGdlviJ5L2c4uPoEKfFcxs=; b=XRymii+hlLwXW7vEFCu2QO+dTZrdD4gMlWWxF2SYGKxFKvUKsU2SGQ8KRYJoJrg3C7 B881Wngxy2R5KatFEvC0lafl5TsdVPILq8RHnkSMWFN5E/9VUNw8YaHOJb1nXutyMEEW B6iAgPQy6V+T7k5/oinMuFHrusHJgGo+Va7mQxdhxxVIKZSOkDVog7C8X2sgCzECxbFQ l49j6Oev2nA+Kn0cRxH5iA+OiJOni4k76Z0JS6OmAR2eI/y7Mlp/4lfbnmFdySlfR/vN KkGpJPsudjk147U+B6fthEG/AWr80MqrKP8D1KqnXL+06ibaGMwP4Rckuoq8w0dMBWWf sQSQ==
X-Gm-Message-State: APjAAAUvti5dHPWTuIvq6WU2F8IeFZ3QjtCJxqrnEmK6o2L/UGxbcj1u MxJwdg+zSb1qqRe54vVKsfev1SMCIA+2xCP/TlJ+LQ==
X-Google-Smtp-Source: APXvYqyk9aNB/42K5qGpCZlfIFtDHKYMossSDHqP88sag6HT429+QFkTCkHMkLtaAtj2Bzg1Ypjb09Re4ek+NBuP3NQ=
X-Received: by 2002:ab0:5a8f:: with SMTP id w15mr230906uae.91.1575307998685; Mon, 02 Dec 2019 09:33:18 -0800 (PST)
MIME-Version: 1.0
References: <CAG9+TpZgm1zbW2Zq9oPCprmgUr-Rh29RTmGyEcVfE+T-i1WvaA@mail.gmail.com> <CAJ_4DfTk11C60LsRhv8Fr_jypxQiweyx0hQnumY_fQHBey-42Q@mail.gmail.com>
In-Reply-To: <CAJ_4DfTk11C60LsRhv8Fr_jypxQiweyx0hQnumY_fQHBey-42Q@mail.gmail.com>
From: Ian Swett <ianswett@google.com>
Date: Mon, 02 Dec 2019 12:33:07 -0500
Message-ID: <CAKcm_gN-tWO3LUTdFzqwqWg3=t3NmJn0wir=qhJ=yOHH94mbbA@mail.gmail.com>
Subject: Re: How to avoid kernel's receive buff full , then UDP packet dropped by kernel?
To: Ryan Hamilton <rch=40google.com@dmarc.ietf.org>
Cc: Jiuhai Zhang <jiuhai.zhang@gmail.com>, IETF QUIC WG <quic@ietf.org>
Content-Type: multipart/alternative; boundary="000000000000aceb410598bbf9c7"
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/RSz2fmROeSw5d1silB-TPv24BKQ>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 02 Dec 2019 17:33:23 -0000

I would advise increasing the value from the default, which is commonly
16k.  1MB seems like a good value, experimentally.

On Fri, Nov 29, 2019 at 6:52 PM Ryan Hamilton <rch=
40google.com@dmarc.ietf.org> wrote:

> On the server side of a QUIC connection, the UDP socket (and hence the
> kernel receive buffer) is shared across multiple connections. As such it'd
> be incredibly complicated to come up with a system to handle this and would
> likely result in unnecessarily throttling throughput. On the client side
> (where there is once QUIC connection per socket) this is unlikely to happen
> if the receive buffer is larger than the BDP of the link. Of course, in
> reality, if the application is reading from the socket frequently enough,
> this is likely a non-issue.
>
> On Fri, Nov 29, 2019 at 4:53 AM Jiuhai Zhang <jiuhai.zhang@gmail.com>
> wrote:
>
>> Is there any good idea to solve this? How can we do flow control like TCP
>> receive window size? User space can not get how many receive buff is used
>>
>