From nobody Fri Mar 5 02:17:19 2021 Return-Path: X-Original-To: tsvwg@ietfa.amsl.com Delivered-To: tsvwg@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0DDDB3A22F1 for ; Fri, 5 Mar 2021 02:17:17 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.433 X-Spam-Level: X-Spam-Status: No, score=-1.433 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, NICE_REPLY_A=-0.001, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=bobbriscoe.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QWqgrrPOZ1bD for ; Fri, 5 Mar 2021 02:17:13 -0800 (PST) Received: from mail-ssdrsserver2.hosting.co.uk (mail-ssdrsserver2.hosting.co.uk [185.185.84.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 542133A22F0 for ; Fri, 5 Mar 2021 02:17:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bobbriscoe.net; s=default; h=Content-Type:In-Reply-To:MIME-Version:Date: Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=t1IeR9WCg9Kmfgbe69GCxqbXy5QhRvtKxfP2rtMxMko=; b=Z5gsfjv8HkzUU8edmTzBBeI9T cgzbahQ4Ct/fvBRazm54zlcp/uSnyknk8J4dGWL0h/ZNKhTHwAhZ0jTnRpHzstDhhWFAZv6RI5w2+ Wg18SLsnNb3y9BMh/r/5gqBVgch+pAWKoZeyslZU8miBhdaERUUnlROgZwIxNLXrFpmoaPZljdRfj +a3WtrCGxcKIJbpiRrYV29GPF2Nr1oA+OU+ecBbiEmABr3rU/5zbbo7Q3CFA0ulyCAHtS7MX5I8eH 7KuLmUTI/8ObRtO/C5sqOnZhU4IVBjixM0bvPgc/b54JwFLYmYP9g1cz6XJqamN4+CgSjF4JF0Pbv z7GKR8weg==; Received: from 67.153.238.178.in-addr.arpa ([178.238.153.67]:54716 helo=[192.168.1.11]) by ssdrsserver2.hosting.co.uk with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1lI7WB-0001Ga-56; Fri, 05 Mar 2021 10:17:11 +0000 To: Ingemar Johansson S , "koen.de_schepper@nokia.com" , "G.White@CableLabs.com" Cc: "tsvwg@ietf.org" References: From: Bob Briscoe Message-ID: <52c53955-cc84-4fcc-fbbe-f01f3377b19c@bobbriscoe.net> Date: Fri, 5 Mar 2021 10:17:10 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------BAE4E54285F729007AE26CA4" Content-Language: en-GB X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - ssdrsserver2.hosting.co.uk X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bobbriscoe.net X-Get-Message-Sender-Via: ssdrsserver2.hosting.co.uk: authenticated_id: in@bobbriscoe.net X-Authenticated-Sender: ssdrsserver2.hosting.co.uk: in@bobbriscoe.net X-Source: X-Source-Args: X-Source-Dir: Archived-At: Subject: Re: [tsvwg] Comments on draft-ietf-tsvwg-aqm-dualq-coupled-13 X-BeenThere: tsvwg@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Transport Area Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Mar 2021 10:17:17 -0000 This is a multi-part message in MIME format. --------------BAE4E54285F729007AE26CA4 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Ingemar, I "lost" your useful review below until now (I tag 'ToDo' emails, but it had got too far back in my queue). See inline tagged [BB] On 07/12/2020 10:05, Ingemar Johansson S wrote: > > Hi > > I have read through draft-ietf-tsvwg-aqm-dualq-coupled-13. > > In general I find the document well written with just a few > remarks/comment/questions listed below. > > + One thing that strikes me is that packet pacing is not mentioned in > the draft. Packet pacing is implemented in Prague, BBRv2 and SCReAM > and the obvious benefit is that packet bursts from individual flows > become a smaller problem. On the other hand DCTCP does not inherently > assume packet pacing although it can be enabled and is highly > recommended in e.g. HULL. There can also be cases where packet pacing > is not preferable as it can increase e2e latency, one such example is > very low latency streaming of video where it can be beneficial to just > burst the video frames to save some serialization delay in links with > high statistical multiplexing. > [BB] You are right that pacing is necessary at the sender. But this draft is about the AQM, not the host algorithms. For their requirement to limit bursts see the last bullet in https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-13#section-4.3 There is also a section about limiting bursts from the link technology that underlies an L4S AQM here: https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-11#section-5.5 This is not in the DualQ draft because it applies generally to any L4S AQM. However, you've made me realize there should be a statement in the DualQ draft saying that a DualQ Coupled AQM MUST comply with all the prerequisite behaviours of an L4S network node in the network section of ecn-l4s-id That's an important omission. Thank you for catching that. I've added it to my local copy for the next rev. > Does the L4S marker assume packet pacing?, I don’t think so and > perhaps this should not be elaboratet upon in this draft as packet > pacing is a general networking topic bigger than DualQ ?. > [BB] The marker in the AQM just marks what it finds. So if a source does not pace and therefore bursts cause the queue to keep exceeding its threshold, it will just mark the packets over the threshold. It doesn't know why though. > + Section 1.4 mentions a VR use case “VR goggles was remotely > receiving a feed from a 360-degree camera in a racing car”. While I > find the football demo (a real eye opener), I don’t seem to find the > VR demo ? > [BB] I'm afraid no-one recorded a video of the demo :( It also included numerous other applications all running over the same 40Mb/s broadband access link, all with extremely low latency. There's a short (2pp) paper about the demo at MMSYS'16 here: https://bobbriscoe.net/pubs.html#uld4all-demo > + Page 15; Quote “As a consequence, DualPI2 has attracted more > development and evaluation attention than  Curvy RED, leaving the > Curvy RED design incomplete and not so fully evaluated.” This makes > sense in 2020 but in 10 years from now?. For anybody familiar with > IETF  it will be known  that this is discussion text but for people > not so familiar it may not be the case. I have over the years seen > discussion text in e.g. RFC3168 taken completely outside its context, > it is quite apparent that this text reflected the status when one of > my kids was just a little toddler but still it can be brought up 😊 > [BB] I've added "at the time of writing" to my local copy. Thanks. > + Section 2.5.1: “work-conserving”. Luckily I know what the term is, > it was not the case 10 years ago though. Is the reader assumed to know > what it is ? I know a simple search on the internet gives the answer > so it perhaps does not need an explanation > [BB] It's a term that an AQM implementer should know, and as you say, a quick look-up will fix that otherwise. > + Section 2.5.1.1 : “if the packet is ECT(0), the L AQM SHOULD apply > CE-marking using a probability appropriate to Classic congestion > control and appropriate to the target delay in the L queue” My first > though was .. how is this done but then I believe I get it, does this > mean that the pseudo code needs to compute a p’_LC = p’L^2 for this > purpose and apply marking with probability p’_LC on packets with ECT(0) ? > [BB] Some of the scenarios in this "Unexpected Cases" section, including this "SHOULD", are rather corner-cases. It might not be possible for an ECT0 packet to appear in the L queue, then you wouldn't need to implement these. But we included this section in case the operator hand-crafted classifiers to allow some cases. For instance, in the case of DOCSIS, low latency DSCPs are classified into the L queue, irrespective of the ECN field, so for simplicity ECT(0) packets in the L queue are just treated as Not-ECT. Low latency DSCPs with Not-ECT are not dropped unless per-flow queue protection redirects them into the C queue, where the Classic AQM might introduce some drop. If the operator did want to mark ECT0 packets, I think you've sort-of got the right idea. However, once you've squared p’L, you still have to take max(p’L, p_CL) to couple across any congestion in the C queue. Also strictly, before you square, you should take a moving average of p’L so that it's like a Classic AQM (in fact this would be RED, but with the queue measured in time units). We took the view that this is all rather too complex for a likely corner case, which is why it says "SHOULD", so simpler mechanisms can "cut off the corners". > + Section 4.1.1. : Scheduling weights , is there any studies on > weights such as 1/8 ? > [BB] It's not that sensitive to the weight, so 1/8 would be fine. The idea of recommending 1/16 was purely to ensure the scheduler rarely determines the shares (only if the ratio of long-running L to C flows exceeds 16:1). So with a weight of 1/8, if you have >8 L4S flows to 1 Classic, the scheduler rather than the congestion controls will start to determine capacity. But the differences are still tiny. Specifically, with a weight of 1/8, with L:C flow numbers at 10:1, each L4S flow will get 7/8/10 = 8.75%, instead of what you'd expect for 11 flows: 1/11 = 9.1%. With more extreme numbers, e.g. 20:1, each L4S flow would get 7/8/20 = 4.375%, instead of 1/21 = 4.76%. So, as you can see, the 'error' is tiny. Of course the single C flow would always get 1/8 = 12.5%, which is rather more than you'd expect. But even tho you can get larger positive errors like this, it ensures you don't get large negative errors, even with really extreme flow numbers. > + Appendix A : The pseudocode looks very good and is easy to > understand with the help from the surrounding text. Notice however > that there is a mix of code conventions, for instance Tupdate, > RTT_max. Looking back at RFC8298 I see that the pseudo code there has > variable names such as bytes_in_flight and constants such as MIN_CWND. > [BB] You're right. It would help if the constants were in CAPS. I'm not going to do that now tho - added to my ToDo list. I'd need to be awake to ensure I didn't make mistakes, 'cos I don't have a pseudocode compiler to check my work ;) > + Appendix B: Question, will this section be relevant in the future?, > it appears that you have focused mainly on the PI2 that is explained > in Appendix A. Does Curvy RED have some benefits over PI2 that makes > it relevant to have it in the draft? > [BB] Curvy RED has been implemented in Broadcom's Strata DNX chipset. I haven't seen any performance results tho. I guess Curvy RED is more familiar to those who know RED. I don't think PI2 is any more complex than RED though. > + Nit.. The bullet list sometimes begin with a capital letter e.g > ‘If..’ and sometimes not (‘if..’) > [BB] That's usually OK and deliberate. When the bullets are whole sentences they start with an upper case. When they continue the sentence before the list, they start with lower case. But I did find a couple of lists that were incorrectly lower cased. I've fixed in my local copy. I notice one list with one lower and one upper, but that's because the upper is a defined word. Cheers, and sorry again for appearing to have ignored all the work you'd put into this review. Bob > /Ingemar > > ================================ > > Ingemar Johansson  M.Sc. > > Master Researcher > > Ericsson Research > > RESEARCHER > > GFTL ER NAP NCM Netw Proto & E2E Perf > > Labratoriegränd 11 > > 977 53, Luleå, Sweden > > Phone +46-1071 43042 > > SMS/MMS +46-73 078 3289 > > ingemar.s.johansson@ericsson.com > > www.ericsson.com > > Talk about a dream, try to make it real > > Bruce Springsteen > > ================================= > -- ________________________________________________________________ Bob Briscoe http://bobbriscoe.net/ --------------BAE4E54285F729007AE26CA4 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit Ingemar,

I "lost" your useful review below until now (I tag 'ToDo' emails, but it had got too far back in my queue).
See inline tagged [BB]

On 07/12/2020 10:05, Ingemar Johansson S wrote:

Hi

I have read through draft-ietf-tsvwg-aqm-dualq-coupled-13.

In general I find the document well written with just a few remarks/comment/questions listed below.

 

+ One thing that strikes me is that packet pacing is not mentioned in the draft. Packet pacing is implemented in Prague, BBRv2 and SCReAM and the obvious benefit is that packet bursts from individual flows become a smaller problem. On the other hand DCTCP does not inherently assume packet pacing although it can be enabled and is highly recommended in e.g. HULL. There can also be cases where packet pacing is not preferable as it can increase e2e latency, one such example is very low latency streaming of video where it can be beneficial to just burst the video frames to save some serialization delay in links with high statistical multiplexing.


[BB] You are right that pacing is necessary at the sender. But this draft is about the AQM, not the host algorithms. For their requirement to limit bursts see the last bullet in https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-13#section-4.3

There is also a section about limiting bursts from the link technology that underlies an L4S AQM here:
https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-11#section-5.5
This is not in the DualQ draft because it applies generally to any L4S AQM. However, you've made me realize there should be a statement in the DualQ draft saying that a DualQ Coupled AQM MUST comply with all the prerequisite behaviours of an L4S network node in the network section of ecn-l4s-id
That's an important omission. Thank you for catching that. I've added it to my local copy for the next rev.

Does the L4S marker assume packet pacing?, I don’t think so and perhaps this should not be elaboratet upon in this draft as packet pacing is a general networking topic bigger than DualQ ?.


[BB] The marker in the AQM just marks what it finds. So if a source does not pace and therefore bursts cause the queue to keep exceeding its threshold, it will just mark the packets over the threshold. It doesn't know why though.

 

+ Section 1.4 mentions a VR use case “VR goggles was remotely receiving a feed from a 360-degree camera in a racing car”. While I find the football demo (a real eye opener), I don’t seem to find the VR demo ?


[BB] I'm afraid no-one recorded a video of the demo :(
It also included numerous other applications all running over the same 40Mb/s broadband access link, all with extremely low latency.
There's a short (2pp) paper about the demo at MMSYS'16 here: https://bobbriscoe.net/pubs.html#uld4all-demo


 

+ Page 15; Quote “As a consequence,  DualPI2 has attracted more development and evaluation attention than  Curvy RED, leaving the Curvy RED design incomplete and not so fully evaluated.” This makes sense in 2020 but in 10 years from now?. For anybody familiar with IETF  it will be known  that this is discussion text but for people not so familiar it may not be the case. I have over the years seen discussion text in e.g. RFC3168 taken completely outside its context, it is quite apparent that this text reflected the status when one of my kids was just a little toddler but still it can be brought up 😊 


[BB] I've added "at the time of writing" to my local copy. Thanks.

 

+ Section 2.5.1: “work-conserving”. Luckily I know what the term is, it was not the case 10 years ago though. Is the reader assumed to know what it is ? I know a simple search on the internet gives the answer so it perhaps does not need an explanation

 


[BB] It's a term that an AQM implementer should know, and as you say, a quick look-up will fix that otherwise.

+ Section 2.5.1.1 : “if the packet is ECT(0), the L AQM SHOULD apply CE-marking using a probability appropriate to Classic congestion control and appropriate to the target delay in the L queue” My first though was .. how is this done but then I believe I get it, does this mean that the pseudo code needs to compute a p’_LC = p’L^2 for this purpose and apply marking with probability p’_LC on packets with ECT(0) ?


[BB] Some of the scenarios in this "Unexpected Cases" section, including this "SHOULD", are rather corner-cases. It might not be possible for an ECT0 packet to appear in the L queue, then you wouldn't need to implement these. But we included this section in case the operator hand-crafted classifiers to allow some cases. For instance, in the case of DOCSIS, low latency DSCPs are classified into the L queue, irrespective of the ECN field, so for simplicity ECT(0) packets in the L queue are just treated as Not-ECT. Low latency DSCPs with Not-ECT are not dropped unless per-flow queue protection redirects them into the C queue, where the Classic AQM might introduce some drop.

If the operator did want to mark ECT0 packets, I think you've sort-of got the right idea. However, once you've squared p’L, you still have to take max(p’L, p_CL) to couple across any congestion in the C queue. Also strictly, before you square, you should take a moving average of p’L so that it's like a Classic AQM (in fact this would be RED, but with the queue measured in time units). We took the view that this is all rather too complex for a likely corner case, which is why it says "SHOULD", so simpler mechanisms can "cut off the corners".

 

+ Section 4.1.1. : Scheduling weights , is there any studies on weights such as 1/8 ?


[BB] It's not that sensitive to the weight, so 1/8 would be fine. The idea of recommending 1/16 was purely to ensure the scheduler rarely determines the shares (only if the ratio of long-running L to C flows exceeds 16:1). So with a weight of 1/8, if you have >8 L4S flows to 1 Classic, the scheduler rather than the congestion controls will start to determine capacity. But the differences are still tiny.

Specifically, with a weight of 1/8, with L:C flow numbers at 10:1, each L4S flow will get 7/8/10 = 8.75%, instead of what you'd expect for 11 flows: 1/11 = 9.1%.
With more extreme numbers, e.g. 20:1, each L4S flow would get 7/8/20 = 4.375%, instead of 1/21 = 4.76%.
So, as you can see, the 'error' is tiny. Of course the single C flow would always get 1/8 = 12.5%, which is rather more than you'd expect. But even tho you can get larger positive errors like this, it ensures you don't get large negative errors, even with really extreme flow numbers.


 

+ Appendix A : The pseudocode looks very good and is easy to understand with the help from the surrounding text. Notice however that there is a mix of code conventions, for instance Tupdate, RTT_max. Looking back at RFC8298 I see that the pseudo code there has variable names such as bytes_in_flight and constants such as MIN_CWND.


[BB] You're right. It would help if the constants were in CAPS. I'm not going to do that now tho - added to my ToDo list. I'd need to be awake to ensure I didn't make mistakes, 'cos I don't have a pseudocode compiler to check my work ;)

 

+ Appendix B: Question, will this section be relevant in the future?, it appears that you have focused mainly on the PI2 that is explained in Appendix A. Does Curvy RED have some benefits over PI2 that makes it relevant to have it in the draft?


[BB] Curvy RED has been implemented in Broadcom's Strata DNX chipset. I haven't seen any performance results tho. I guess Curvy RED is more familiar to those who know RED. I don't think PI2 is any more complex than RED though.

 

+ Nit.. The bullet list sometimes begin with a capital letter e.g ‘If..’ and sometimes not (‘if..’)


[BB] That's usually OK and deliberate. When the bullets are whole sentences they start with an upper case. When they continue the sentence before the list, they start with lower case. But I did find a couple of lists that were incorrectly lower cased. I've fixed in my local copy.

I notice one list with one lower and one upper, but that's because the upper is a defined word.

Cheers, and sorry again for appearing to have ignored all the work you'd put into this review.



Bob

 

/Ingemar

================================

Ingemar Johansson  M.Sc.

Master Researcher

 

Ericsson Research

RESEARCHER

GFTL ER NAP NCM Netw Proto & E2E Perf

Labratoriegränd 11

977 53, Luleå, Sweden

Phone +46-1071 43042

SMS/MMS +46-73 078 3289

ingemar.s.johansson@ericsson.com

www.ericsson.com

 

Talk about a dream, try to make it real

                  Bruce Springsteen

=================================

 


-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/
--------------BAE4E54285F729007AE26CA4--