Re: [DNSOP] Lars Eggert's Discuss on draft-ietf-dnsop-dns-tcp-requirements-13: (with DISCUSS and COMMENT)

"Wessels, Duane" <dwessels@verisign.com> Mon, 29 November 2021 23:53 UTC

Return-Path: <dwessels@verisign.com>
X-Original-To: dnsop@ietfa.amsl.com
Delivered-To: dnsop@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9C43E3A0B4C; Mon, 29 Nov 2021 15:53:55 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.099
X-Spam-Level:
X-Spam-Status: No, score=-2.099 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=verisign.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bCxn5Kae8R7L; Mon, 29 Nov 2021 15:53:50 -0800 (PST)
Received: from mail4.verisign.com (mail4.verisign.com [69.58.187.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3BAFF3A0B41; Mon, 29 Nov 2021 15:53:50 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=verisign.com; l=9730; q=dns/txt; s=VRSN; t=1638230031; h=from:to:cc:date:message-id:references:in-reply-to: content-id:content-transfer-encoding:mime-version:subject; bh=o09SdiEOemOfg3TnTV4B7g5SwWxaBK5wjnJ4UkBWvAA=; b=C5LvaGBvPvTdhHqM4cmTZL83631CME/LkiOc1cWY5TVMMzt6R7ChsnoN 9dRaWFkbp68LFCj7X5lXxgtqjffvcj1KU7u8MwvtEo9wdtoYeW4ngciVA SmXORTSYVMFCbwmR1lAtCx59v7Bh0C7kyCoAHhw3OZheF/p8Ib4RpQ8nE 87hY85SV8eUnyb/nfniSPLDtL23XeQE9MR99VPV5ePzjCYjHaYr/GvEv8 Np83qHjJDcHAz7pyFA8TXDAxDTssd3xiJfM0a79dBEx1Q69zM0uYZVqid D1b2tTzJMDQ0abjMpHMz4faW7Ld2UlXsui/gz8OxjBQADwhzNwvKlvZLR w==;
IronPort-SDR: phUTQGjnHJUjsqBMas3j6pX7MNpgItyJTiD3XYf8+yzmuNZmULNV+Q9nlzQCRQzpZjmD4XIo8H xu0EgNqorzRNmac9vhhPa2+84qtALqvjkAbi//VUJ909xnj7XgLokHDkB1eQ77jT5VbSj23I5F VZmLVr/RgVtJzgB5w7PWUZ6sxhBkkXkuq7E8Zp2YaXKyUYPDg6I6EXBJDa1xmclcSGuYUWLOdd hoiFQeP/LUzNcU5BUaqHwO78Gz2Sle/ItNIqZJf0OyP9nDuhiiyLWAyZAgvRqgPUdKeWErqf+e FNA=
IronPort-Data: A9a23:08/U06LfVUK0sz1oFE+RyZQlxSXFcZb7ZxGr2PjKsXjdYENS3jwBy jBKWTuGPP+KZTD1KIwnaN/kpkkGupbczIJrQAJorCE8RH908seUXt7xwmUcn8+xwmwvaGo9s q3yv/GZdJhcokcxJX5BC5C5xZVG/fjgqoHUVaiUZUideSc+EH140Es5yrZj6mJVqYPR7z2l6 IuaT/L3ZQfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHpxdGlOjKmVi8k5Wc M6YpF2x1juxEx4FVIv5wu6jGqEAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqho8JB4 e5Mv7GKEjgKEK/Swv8ZeT1lDHQrVUFG0OevzXmXm/a1lnLgXku0mbNwB0YsJcsR9qBpG3pIs /cfLVjhbDja36Tvn+n9E7Q3wJhyRCXoFNp3VnVIwS7UFu0rRYvrXajQ5MRZ0zF2jcdLdRrbT 5NANWcxMEuZC/FJElY8Oo5hw92Dv3fyUi0CrkCl4poy7FGGmWSd15CoarI5YOeiQcxPhW6Vq W3L5Xi/BQsVXPSexCGK2nOhmuGJmjn0MKoeDrS26rtrjUGdg2AVExoOEFe9urylgVb7Qd9GN k0O9y8jq7Qp3E2mUte7WAe3yFaIuAUbQ59bE+Q78hqly6fI7UCeHGdsZjJHc9s+nM47WTJs0 UWG9/vyGC5wt6eRRW2c+rq8oja7OCxTJmgHDRLoViMP+d+6v4c+nkqVC819Cuiwj8awEza2y SqM9W4gna4Vy8UM0s1X4Gz6vt5lnbCRJiZd2+kddjvNAt9RDGJ9W7GV1A==
IronPort-HdrOrdr: A9a23:Df4PaK3iY2ioHcT2EuL9jwqjBG8kLtp133Aq2lEZdPUzSL38qy nOpoV46faaslYssR0b9+xoW5PufZq0z/cc3WB7B8bAYOCJggqVBbAnw4fkzybpBiHyssVMvJ 0NT4FOTPn9F0Jzg8q/wgWpeuxL/PC3tISln/3XwXsodxxtcK0I1WpEIxyWCVJ7XzNLApcFFJ 6Rj/Atmwad
X-IronPort-AV: E=Sophos;i="5.87,273,1631592000"; d="scan'208";a="11456471"
Received: from BRN1WNEX01.vcorp.ad.vrsn.com (10.173.153.48) by BRN1WNEX02.vcorp.ad.vrsn.com (10.173.153.49) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.7; Mon, 29 Nov 2021 18:53:48 -0500
Received: from BRN1WNEX01.vcorp.ad.vrsn.com ([fe80::a89b:32d6:b967:337d]) by BRN1WNEX01.vcorp.ad.vrsn.com ([fe80::a89b:32d6:b967:337d%4]) with mapi id 15.01.2375.007; Mon, 29 Nov 2021 18:53:48 -0500
From: "Wessels, Duane" <dwessels@verisign.com>
To: Lars Eggert <lars@eggert.org>
CC: The IESG <iesg@ietf.org>, "draft-ietf-dnsop-dns-tcp-requirements@ietf.org" <draft-ietf-dnsop-dns-tcp-requirements@ietf.org>, "dnsop-chairs@ietf.org" <dnsop-chairs@ietf.org>, "dnsop@ietf.org" <dnsop@ietf.org>, Suzanne Woolf <suzworldwide@gmail.com>
Thread-Topic: [EXTERNAL] Lars Eggert's Discuss on draft-ietf-dnsop-dns-tcp-requirements-13: (with DISCUSS and COMMENT)
Thread-Index: AQHX5Xxa1WHeYbA7FU2LSxLF5LNQ7Q==
Date: Mon, 29 Nov 2021 23:53:48 +0000
Message-ID: <AB8A981D-4923-4D58-8035-3708E4C991A1@verisign.com>
References: <163524814397.6773.7925615506385048342@ietfa.amsl.com>
In-Reply-To: <163524814397.6773.7925615506385048342@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3608.120.23.2.7)
x-originating-ip: [10.170.148.18]
Content-Type: text/plain; charset="us-ascii"
Content-ID: <3DCE4C9F16CA1044834DE93834800BBD@verisign.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Archived-At: <https://mailarchive.ietf.org/arch/msg/dnsop/kNayvbDkgd28FQntzLZ2b_U3c58>
Subject: Re: [DNSOP] Lars Eggert's Discuss on draft-ietf-dnsop-dns-tcp-requirements-13: (with DISCUSS and COMMENT)
X-BeenThere: dnsop@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF DNSOP WG mailing list <dnsop.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/dnsop>, <mailto:dnsop-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/dnsop/>
List-Post: <mailto:dnsop@ietf.org>
List-Help: <mailto:dnsop-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/dnsop>, <mailto:dnsop-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 29 Nov 2021 23:53:56 -0000


> On Oct 26, 2021, at 4:35 AM, Lars Eggert via Datatracker <noreply@ietf.org> wrote:
> 
> Caution: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. 
> 
> Lars Eggert has entered the following ballot position for
> draft-ietf-dnsop-dns-tcp-requirements-13: Discuss
> 
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
> 
> 
> Please refer to https://secure-web.cisco.com/1kXvipN4VaKHO3AYxNHSk3_n8uHuI5gukOoaWs_9cG3yENUfEij_EahsVifsuqDvJ87tOMzqfsQvSNVDrlS_-uD93gYFrH-W0nErz-3dDIJpel5Zl7MmuWBdfniJzujsocAV1lsSsYtX8awWBkQ8eb_GhRen4BPENNuz1f9DU8scOGmh_6XvDTl2h2Ut3BN2YuNmDbmfWLYWKn2ljBjy70M3-N-vnFroRv7U3a3Kq-iNDmhKR53kaMlSwzI1NM_rK/https%3A%2F%2Fwww.ietf.org%2Fblog%2Fhandling-iesg-ballot-positions%2F
> for more information about how to handle DISCUSS and COMMENT positions.
> 
> 
> The document, along with other ballot positions, can be found here:
> https://secure-web.cisco.com/1V7ehlyVXKiQcvhjjSAc0dm4x7jce6oRNiVmeJVwrJmYfNJTQCXozSCiVsTTDTMA31OsPaLi5ktIBX_1SJTMwPOMjHkyejN20CFahmTm4V-mFwr3n3zhznbcttOwt49mZNQ24MwFa4SFXIhHivGRuI65YKsZUQDdEJj2ORiTP9kCyMMSw6uGu5eE_JQtlH1M3Q7rqSm33c6SaE0h2NlmjlKGPa6zzXngPE8McI90Hbd47Q3rc4CuANJZLqNdhgNwu/https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-ietf-dnsop-dns-tcp-requirements%2F
> 
> 
> 
> ----------------------------------------------------------------------
> DISCUSS:
> ----------------------------------------------------------------------
> 
> Section 4.2. , paragraph 3, discuss:
>>   Since host memory for TCP state is a finite resource, DNS clients and
>>   servers MUST actively manage their connections.  Applications that do
>>   not actively manage their connections can encounter resource
>>   exhaustion leading to denial of service.  For DNS, as in other
>>   protocols, there is a tradeoff between keeping connections open for
>>   potential future use and the need to free up resources for new
>>   connections that will arrive.
> 
> For it to contain a MUST-level requirement, this section needs to give a lot
> more concrete guidance on what it means to "actively" manage connections. Most
> operating systems by default impose some application limits that usually
> effectively prevent DOS or other resource exhaustion issues. Is the intent here
> that DNS implementations need to do more? If so, what?

I can easily be convinced that SHOULD is more appropriate than MUST here.  

I dont necessarily agree that operating systems alone do a very good job
of preventing DOS conditions.  It is possible that Im not up-to-date on
the latest and greatest in terms of operating system features, but I think
historically applications have fared better when they manage their own
connections.  For example, can we realistically expect the OS to know which
idle connections should be closed?

> 
> 
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
> 
> Section 2.4. , paragraph 1, comment:
>> 2.4.  Fragmentation and Truncation
> 
> Fragmentation and IP fragments getting dropped is one reason for needing more
> retries with EDNS(0). But IIRC, a larger contributing factor is that EDNS(0)
> doesn't detect or recover from loss of any UDP packets making up the overall
> message. That means that a normal packet loss (due to congestion or other
> reasons) amplifies into loss of the entire DNS message.

This new paragraph has been added:

   Note that a receiver is unable to differentiate between packets lost
   due to congestion and packets (fragments) intentionally dropped by
   firewalls or middleboxes.  Over network paths with non-trival amounts
   of packet loss, larger, fragmented DNS responses are more likely to
   never arrive and time out compared to smaller, unfragmented
   responses.  Clients might be misled into retrying queries with
   different EDNS(0) UDP packet size values for the wrong reason.

> 
> Section 3. , paragraph 1, comment:
>> 3.  DNS over TCP Requirements
> 
> While the history preceding this section is interesting for context, I think
> moving this section up would increase readability significantly.

Okay with me.  I would like to hear from the Chairs.

> 
> Section 4.2. , paragraph 3, comment:
>>   DNS server software SHOULD provide a configurable limit on the total
>>   number of established TCP connections.  If the limit is reached, the
>>   application is expected to either close existing (idle) connections
>>   or refuse new connections.  Operators SHOULD ensure the limit is
>>   configured appropriately for their particular situation.
> 
> Again, the OS generally already imposes limits. Why recommend that DNS
> implementations self-impose other (lower?) limits?
Perhaps the below text is better?  It is directed more to operators (the
intended audience) rather than implementors, and acknowledges the reality
that implementations already have such limits today.


   Operators of DNS server software SHOULD be aware that operating
   system and application vendors MAY impose a limit on the total number
   of established connections.  These limits may be designed to protect
   against DDoS attacks or performance degradation.  Operators SHOULD
   understand how to increase these limits if necessary, and the
   consequences of doing so.


> 
> Section 4.2. , paragraph 3, comment:
>>   DNS server software MAY provide a configurable limit on the number of
>>   established connections per source IP address or subnet.  This can be
>>   used to ensure that a single or small set of users cannot consume all
>>   TCP resources and deny service to other users.  Operators SHOULD
>>   ensure this limit is configured appropriately, based on their number
>>   and diversity of users.
> 
> This limit only begins to establish fairness once the overall number of
> permitted connections is reached. Until that point, it possibly limits client
> performance without any benefit.

I suppose this depends on how it is implemented and there are lessons to be
learned from the world of IP QOS.  

I surveyed open source DNS server code and their developers and found that only
one implements this limit and ships with no limit by default.  

Perhaps this current (updated) paragraph is better:

   DNS server software MAY provide a configurable limit on the number of
   established connections per source IP address or subnet.  This can be
   used to ensure that a single or small set of users cannot consume all
   TCP resources and deny service to other users.  Note, however, that
   if this limit is enabled, it possibly limits client performance while
   leaving some TCP resources unutilized.  Operators SHOULD be aware of
   these tradeoffs and ensure this limit, if configured, is set
   appropriately based on the number and diversity of their users, and
   whether users connect from unique IP addresses or through a shared
   Network Address Translator [RFC3022].



> 
> Section 4.2. , paragraph 3, comment:
>>   DNS server software SHOULD provide a configurable timeout for idle
>>   TCP connections.  For very busy name servers this might be set to a
>>   low value, such as a few seconds.  For less busy servers it might be
>>   set to a higher value, such as tens of seconds.
> 
> Ditto.

In this case all of the open source implementations I surveyed have this 
limit enabled by default.  


> 
> Section 4.2. , paragraph 3, comment:
>>   DNS server software MAY provide a configurable limit on the number of
>>   transactions per TCP connection.
> 
> What does that limit protect against?

proposed new text:

   DNS server software MAY provide a configurable limit on the number of
   transactions per TCP connection.  This can help protect against
   unfair connection use (e.g., not releasing connection slots to other
   clients) and network evasion attacks.


> 
> Section 4.2. , paragraph 2, comment:
>>   Similarly, DNS server software MAY provide a configurable limit on
>>   the total duration of a TCP connection.
> 
> What does that limit protect against?

Proposed new text:

   Similarly, DNS server software MAY provide a configurable limit on
   the total duration of a TCP connection.  This can help protect
   against unfair connection use, slow read attacks, and network evasion
   attacks.

> 
> Section 4.5. , paragraph 3, comment:
>>   Most open source DNS server implementations provide a configurable
>>   limit on the total number of established connections.  Default values
>>   range from 20 to 150.  In most cases, where the majority of queries
>>   take place over UDP, 150 is a reasonable limit.  For services or
>>   environments where most queries take place over TCP or TLS, 5000 is a
>>   more appropriate limit.
> 
> 20-150 is orders of magnitude less than I expected. Even 5000 seems very low,
> given the capabilities of even low-end servers.


Im not sure what else to say.  20-150 is what we find in current
implementations, and I suspect it must be sufficient for most use cases
(today at least).  I find it difficult to make recommendations for the
range of systems where DNS software can run (from large & expensive servers
on down to the raspberry pi and home routers).

DW