Re: [tcpm] WGLC for draft-ietf-tcpm-prr-rfc6937bis

Yuchung Cheng <> Tue, 08 November 2022 22:25 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 1877FC1522CF for <>; Tue, 8 Nov 2022 14:25:38 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -17.608
X-Spam-Status: No, score=-17.608 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id mfsEfeB2fbbC for <>; Tue, 8 Nov 2022 14:25:34 -0800 (PST)
Received: from ( [IPv6:2a00:1450:4864:20::32c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by (Postfix) with ESMTPS id 0ACC9C14CE23 for <>; Tue, 8 Nov 2022 14:25:33 -0800 (PST)
Received: by with SMTP id t4so9669278wmj.5 for <>; Tue, 08 Nov 2022 14:25:33 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=lTd+PZJ3KXHAc2pCixAUG6xRBAZQbD1N/Ua2qzfq/4M=; b=mBX/n3NlpTUBOI3l8mFrBuVRSXFM9OPEQ4JeM4m/difkw+/pEYkaCUIDOE+eicgv+b O5YxyiuPC45gOX/EZooCo3M/qhlokT3so/BnEe5gAunvDnLr6ubZr9g/yDjgyUeQ6OwL tCSlBW8StxK4nbZb0zI4cCJMpXE48T+PxMlzdCgnlHzufMPmTX1izBl11I6mUdjGQiAm 1ZeIHhQjOJSlamNSB5Lir2KAxs5oSIyVnCXfEaDNtt060C51IWy74RIAZjrithlOM0T8 bclSo5C0EKjwCAar1dmY6iV2g89W0+4tAM7nrBXzrhbJeorMexJz5TWE4Jq083pdor2g 34CQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=lTd+PZJ3KXHAc2pCixAUG6xRBAZQbD1N/Ua2qzfq/4M=; b=RGAQu3a5QUD24HYrwkUntL9D9zLa6iRldoO912vKxGbNZDalNNJoi3Xr8T1i+e8hqP mHt0y77eqkZU3dX/hpQctreFRQkVWAGIXU/1le927tBvNcNNDfAwk8U+6xwS35i1nDNZ OtoU3EHEf7e8qwyPG7nolnoszyHeuzzwn6SHa2Sh+QB7QU/UcTT3RQUle2m3USNkw/fI hR0T2eT7wEGAlh2U7hX0sLA/n57FCeEl9UWdeV0rcnF9yuqnM8n/fGKXWd12EhGh6HvR /s2AHHi/EiZjrkcg6trcyd1b/6x8OCHWqkB4AnhRhbivEotFputVGxq3R4qWljONM9G0 MAOA==
X-Gm-Message-State: ACrzQf0sZBV2n21aRQ/eWaU1gZQyTexVKGwpNMDQ1nCR5H2wdRVldYJj fxOtn6n99g4UZzxRvGz9OswymwmWO7W1A+kgn7Pldg==
X-Google-Smtp-Source: AMsMyM7mF2poJWtg8IofLEjI+Cw5MJAQhJY+hjQ1iyRTXBydS4Qe1A7G+U/yYEWeMJFent9N/fTV4URzJjqQ0Tqx1xI=
X-Received: by 2002:a05:600c:2116:b0:3cf:69ff:5da2 with SMTP id u22-20020a05600c211600b003cf69ff5da2mr35734718wml.16.1667946332243; Tue, 08 Nov 2022 14:25:32 -0800 (PST)
MIME-Version: 1.0
References: <> <>
In-Reply-To: <>
From: Yuchung Cheng <>
Date: Tue, 08 Nov 2022 14:24:55 -0800
Message-ID: <>
To: Markku Kojo <>
Cc: " Extensions" <>
Content-Type: text/plain; charset="UTF-8"
Archived-At: <>
Subject: Re: [tcpm] WGLC for draft-ietf-tcpm-prr-rfc6937bis
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Tue, 08 Nov 2022 22:25:38 -0000

Sorry for the late reply -- forgot to hit send.

On Fri, Oct 28, 2022 at 9:34 PM Markku Kojo
<> wrote:
> Hi,
> I've reviewed the draft. I think it is good to have this doc published
> and that the draft is in pretty good shape.  However, I have a few
> questions and proposals.
> 1.
> Given that RFC6937 is Experimental with some measurement results using
> the algos of RFC6937 and this draft proposes some changes like the
> "safeACK" heuristic that seem to make good sense I wonder if there are
> any experimental results to back up the positive outcomes that the draft
> articulates?
> I'm asking because the draft kinda indicates that this has not been
> explored but at the same time the text describes the benefits in a
> similar fashion that one expects to read when there is measurement data
> showing the benefits. This is a bit confusing.

Good catch -- yes the SafeAck was experimented in Google CDN and the
result was reported in a SIGCOMM paper (section 5.2.1) cited in the
draft. But we'd explicitly mention this point in section 3 "Changes
>From RFC 6937"

> 2.
> The draft explains how Strong Packet Conservation Bound follows the
> spirit of Van Jacobson's packet conservation principle and that in a
> simple network environment with a tail drop queue  PRR would
> not cause any additional drops. This is all correct for a tail-drop
> setting envisioned in the Appendix A and I agree with the text.
> However, if the bottleneck queue employs AQM, packet dynamics and
> behaviour tends to be different. With (almost) any AQM the dropping
> behaviour changes such that once the queue delay/length has been above a
> threshold (e.g., target in CoDel, MinTh in RED) for a long enough time
> (e.g., interval in CoDel) the AQM enters its main operating region and
> starts dropping packets, continuing dropping until the queue delay/length
> is again below the target.
> With a non-PRR sender cwnd is reduced immediately when FR&FR is entered
> and the sender does not inject more packets for  while, allowing the
> AQM queue to drain below the target (in most cases). However, with a PRR
> sender the sending rate is not reduced immediately and the queue is not
> allowed to drain similarly. This means that the new data packets (and
> additional retransmissions) that are sent after the fast retransmit are
> subject to drops because the AQM is still in its main operating region.
> This behavior has been shown in paper
> that Michael pointed out some while ago when he questioned the douple
> drops he had seen:
> So, I am wondering if any additional measurements exists with PRR over an
> AQM botttleneck. Anyway, IMO this shortcoming should be discussed in the
> draft because it may result in suboptimal behaviour with a PRR sender
> over an AQM bottleneck.

I am not aware of further measurements (beside what have been
discussed) about sub-optimal drops. We can add more text on potential
AQM-induced drops in PRR to be comprehensive, as this is indeed what
has been observed.

> 3.
> Given that PRR is recommended to be used with RACK-TLP, it seems that the
> algo does not address the case where RACK-TLP detects a lost
> retransmission, requiring another rate reduction. Now it seems that
> RACK-TLP/PRR may continue with no further rate reductions all the way
> until all losses in a window of data have been recovered no matter how
> many times a retransmission (or some retransmissions) have been dropped
> during the fast recovery. Shouldn't this be addressed as a serious
> glitch?

Could you point out where the draft mandates no further rate reductions?

> 4.
> It might be useful to specify more precisely the interpretation of
> safeACK when used with RFC 6675 "last resort" retransmissions specified
> in Sec 4, NextSeg () steps (3) and (4). Isn't it possible that an
> arriving Ack does not indicate new losses but a "last resort"
> retransmission would be needed? In such a case the recovery is not
> necessarily making "good progress"?

The "last resort" retransmission happens when previous (3-dupack)
rules fail to identify further segments' losses as an opportunistic
aggression to retain ack clock. So the 3 dupack rule could be wrong
(or too strict when infight is small). "Good progress" or not is
subjective and depends on how reliable the 3-dupack rule is.

Having said that, I agree that it'll be good to factor in specific
scenarios in SafeACK to slow start. Hence we can add another condition
to SafeACK:
SafeACK MAY be false if the specific loss detection algorithm used
indicates it may not be safe to accelerate and slow start despite
existing conditions listed in the RFC. For example, when RFC 6675
NextSeg() uses the "last resort" retransmission.

> Best regards,
> /Markku
> On Wed, 12 Oct 2022, Yoshifumi Nishida wrote:
> > Hi folks,
> >
> > This e-mail announces that we initiate a WGLC for draft-ietf-tcpm-prr-rfc6937bis
> > Please send your feedback to the ML.
> > The WGLC runs untils Oct 28.
> >
> > The URL for the doc is
> > Thanks,
> > --
> > Yoshi
> >
> >
> >_______________________________________________
> tcpm mailing list