Re: [netconf] [yang-doctors] Yangdoctors last call review of draft-ietf-netconf-subscribed-notifications-21

Per Hedeland <> Fri, 25 January 2019 21:33 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 6690F124BF6 for <>; Fri, 25 Jan 2019 13:33:05 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=unavailable autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id Vwu30B9oPldF for <>; Fri, 25 Jan 2019 13:33:03 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 51A751271FF for <>; Fri, 25 Jan 2019 13:33:01 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; t=1548451851; cv=none;; s=arc-outbound20181012; b=gkk2OX3lcl5x3gzBOAflAb9fjnUO8GLdomNjl6cJX1JgjEsaYTWVPyvLq5JsSb5e0BqcHHTvJA3M9 SlnpxZAcSJ0DYbG7s+eOwA7gH31RE+RZ0+nAlW5MECWYMTUZm93phYgDJAwWRtTmgWdGC3QcdnLy9v IOrKNFpIO04cJeYcl65bStX+THKtPZCYuBdbYcvMKhVxNBMyb6Opr9HSchY7YPX/XX7pTFnTzjI35L BsKw89ZymvC8Hta6+ZkudbeBd4tBGaJzPLOZK/AQNdM6XjQm8DZk8fLzcc+H5LKvb4AHW3jc1xbBvh LxAgHK+0LoyF3sc5k1dWiUFNBmn2c4g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed;; s=arc-outbound20181012; h=content-transfer-encoding:content-type:in-reply-to:mime-version:date: message-id:from:references:cc:to:subject:dkim-signature:from; bh=EHMIfVFuJBTZvH/yv3mRVjK/nuGh8zagnFtMWIyEocg=; b=FFiuOCu5YtExyjfZbndUXcGatK/qkKSlHHyKKYp5jNV4E1SscRJmwxL+uOByi9urwXrM9Sqcv6haf 6aNY1PqiQE+iYAAmqB4Tan2C94LDbtxV7yHWLbeq5YjT8OF6NSmPb95DU5WRc9eAlxCeElbOpjrbnO qgIuW1Vk7rre3mEEEDA95lI0SjmKfWs21jehAwso24eMV2aXDi74+cWBLN00rXSYhggFmAHFVJQNE8 v7kVXEjIcoEEhKEcmtvFk4jwsWvaoKQ42iAXmgAtqDM3JF7X8EzywKZdd334SjaPf8cv/uDMHQrso6 itY+ZqkDu14W2L/+8g3IZnlFQ8woGGA==
ARC-Authentication-Results: i=1;; spf=none smtp.remote-ip=; dmarc=none; arc=none header.oldest-pass=0;
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=dkim-high; h=content-transfer-encoding:content-type:in-reply-to:mime-version:date: message-id:from:references:cc:to:subject:from; bh=EHMIfVFuJBTZvH/yv3mRVjK/nuGh8zagnFtMWIyEocg=; b=YFjb4VAZjPayHIZzIg5nR3vrQplv8NAq4QvxhcvVyZu7SIzxZtqizg3/n3gp1e6lGtx4rwu/Gv9uD Mg27KYfVZ1MHRbLBiS0TAALKGZ1dQ2D2tz97tEkALDMvCCaU4qY3YtAGq1wNVGNEoYDNq2yAmcCXGK NWsLlrM5WYJZacKQI/F0FLVdvbxMi8G2mDdx92KHacvojc1L8ekMja+qm3iIagaL/qi3OsWjeIYEQ5 UAUOy0ONnUoVdujxoBFpOpnr85E6TLCf4i+7h1PGzAo+Grd6DYLaOMRYBMHr9oK/q9s/rB8l5ZfgYW y8Hn066KzvsUAreN8Eg/DbecEKzZUOg==
X-MHO-RoutePath: cGVyaGVkZWxhbmQ=
X-MHO-User: 7a1bd7c3-20e8-11e9-a59a-7b143e15dabc
X-Mail-Handler: DuoCircle Outbound SMTP
Received: from (unknown []) by (Halon) with ESMTPSA id 7a1bd7c3-20e8-11e9-a59a-7b143e15dabc; Fri, 25 Jan 2019 21:30:49 +0000 (UTC)
Received: from ( []) by (8.15.2/8.15.2) with ESMTPS id x0PLVrtV086757 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Fri, 25 Jan 2019 22:31:53 +0100 (CET) (envelope-from
To: "Eric Voit (evoit)" <>, Martin Bjorklund <>
Cc: "" <>, "" <>, "" <>
References: <> <> <> <> <> <> <>
From: Per Hedeland <>
Message-ID: <>
Date: Fri, 25 Jan 2019 22:31:53 +0100
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Archived-At: <>
Subject: Re: [netconf] [yang-doctors] Yangdoctors last call review of draft-ietf-netconf-subscribed-notifications-21
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 25 Jan 2019 21:33:05 -0000

On 2019-01-25 22:15, Eric Voit (evoit) wrote:
>> From: Per Hedeland, January 25, 2019 1:20 PM
>> On 2019-01-25 16:41, Eric Voit (evoit) wrote:
>>>> From: Martin Bjorklund, January 25, 2019 10:30 AM
>>>> "Eric Voit (evoit)" <> wrote:
>>>>>> From: Martin Bjorklund, January 25, 2019 3:40 AM
>>>>>> "Eric Voit (evoit)" <> wrote:
>>>>>>>> "Eric Voit (evoit)" <> wrote:
>>>>>>>>>> "Eric Voit (evoit)" <> wrote:
>>>>>>>>>>>> From: Martin Bjorklund, January 24, 2019 9:40 AM
>>>>>>>>>>>> "Eric Voit (evoit)" <> wrote:
>>>>>>>>>>>>>> From: Martin Bjorklund, January 24, 2019 8:17 AM
>>>>>> [...]
>>>>>>>>>>>>>> Thinking some more, what is supposed to happen if the
>>>>>>>>>>>>>> client on the same session sends first an
>>>>>>>>>>>>>> establish-subscription with dscp 42, and then another
>>>>>>>>>>>>>> establish-
>>>>>> subscription with dscp 10?
>>>>>>>>>>>>> This would be allowed.
>>>>>>>>>>>> On linux at least this is a sockopt, i.e., the option applies
>>>>>>>>>>>> to the socket, which means all packets on the session.  So
>>>>>>>>>>>> how is this supposed to be implemented if different messages
>>>>>>>>>>>> on the session should have different dscp values?
>>>>>>>>>>>> Or is the
>>>>>>>>>>>> idea that you send the msg, flush all data from ssh/tls to
>>>>>>>>>>>> tcp, then flush the tcp buffers (not that easy...)?
>>>>>>>>>>>> Even if there's just one establish-subscription with a dscp
>>>>>>>>>>>> value, since it applies to the session it means that all
>>>>>>>>>>>> normal rpcs on this session will get the same dscp value.
>>>>>>>>>>>> It is not clear that this is the intention?
>>>>>>>> Did you miss these questions?
>>>>>>> With NETCONF, one DSCP will apply to single TCP session.  So there
>>>>>>> is no
>>>>>> issue.
>>>>>> The problem is if a client sends two establish-subscriptions with
>>>>>> different values for dscp on the same session.  Or even if the
>>>>>> client sends one
>>>>>> establish-
>>>>>> subscription with som dscp value, then the dscp will be applied to
>>>>>> all packets from other rpcs as well.
>>>>>> ... and even worse, with SSH channels you can have multiple NETCONF
>>>>>> session on the same TCP session, which means that the dscp value
>>>>>> applies to all packets in all NETCONF sessions sharing the same TCP
>>>>>> session.
>>>>>> I don't know what the right thing to do is.  Probably first agree
>>>>>> that this is in fact a problem, then maybe simply document this
>>>>>> somehow.
>>>>> I hadn't thought about multiple NETCONF SSH channels.  I agree
>>>>> something to guide developers is needed here.  The easiest fix is to
>>>>> recommend not supporting feature "dscp" with NETCONF.  (It shouldn't
>>>>> be absolutely prohibited as DSCP related RFC's such as RFC7657 embed
>>>>> text like "a single DSCP should be used for all packets in a TCP
>>>>> connection".)
>>>>> Here is the text I suggest is put into
>>>>> draft-ietf-netconf-netconf-event-notifications, Section 5:
>>>>> "The feature "dscp" SHOULD NOT be supported over NETCONF.  This will
>>>>> avoid the potential for out-of-order packet delivery of the set of
>>>>> all traffic within the TCP session."
>>>> I think this is a bit too limiting.  For configured subscriptions it
>>>> can be perfectly fine, since the server can ensure that there's a
>>>> single TCP session to the receiver in this case.  (I know we don't support
>> configured subscriptions at the moment).
>>>> Perhaps we can say in the SN document:
>>>>     If a server that supports the "dscp" feature cannot guarantee that
>>>>     the only packets sent on an underlying transport session are from
>>>>     the subscription, then it should reject the subscription with a
>>>>     "dscp-unavailable" error.  This will avoid the potential for
>>>>     out-of-order packet delivery of the set of all traffic within the
>>>>     TCP session.
>>> This works for me.    I will put this in Section 2.3, directly after the sentence:
>>>      If the publisher supports the "dscp" feature, then a subscription
>>>      with a "dscp" leaf MUST result in a corresponding [RFC2474] DSCP
>>>      marking being placed within the IP header of any resulting
>>>      notification messages and subscription state change notifications.
>>>    unless I hear any objections on this thread.
>> I'm not sure why the "avoid out-of-order" rationale was presented.
>> There is no order "of the set of all traffic within the TCP session"
>> to be out-of. Regardless of DSCP, SSH peers will send data on the channels
>> (NETCONF sessions) in an order that depends, among other things, on the rate
>> of consumption of that data. They will preserve order on any given channel, but
>> will not consider order with respect to other channels (arguably, not having to
>> do that is the whole point of having flow control at the SSH level).
> Hi Per,
> What you say is absolutely true.  You likely know this, but Martin asked to include the out-of-order rationale because if a TCP session carries traffic consisting of a set of DSCP values, then the network elements between the publisher and receiver might reorder the packets.

Hm, from the e-mail thread, it seems that the issue of reordering was
introduced by you, and Martin's text just copied your text regarding
that issue, while relaxing the requirement from completely forbidding
the use of the DSCP parameter to disallowing it where it could have
undesired effects.

> While the order might be restored at the receiver (per your point), this might also be seen as loss.  The result to the application is that higher priority dscp packets will be seen no faster those with lower priority.  Again you likely know this, but there is more on this general topic in places like RFC-7657, Section 5.2.
> To cover what Martin wants, and reduce the non-normative text, how about the following?...
> "Where TCP is used, a publisher which supports the "dscp" feature SHOULD ensure that a subscription's notification messages are returned within a single TCP transport session where all traffic shares the subscription's requested "dscp" leaf value.   Where this cannot be guaranteed, any "establish subscription" RPC request SHOULD be rejected with a "dscp-unavailable" error."

This seems fine to me. The point, as far as I can see, is that a DSCP
parameter must not be applied to a session (or subscription) that
didn't "ask" for it. This is enough of a rationale IMHO, no need to go
into the details of what the result may or may not be as far as
reordering of packets goes.


> Eric
>> I suggest simply dropping the "This will avoid ..." sentence.
>> --Per
>>> Eric
>>>> I think such text belongs to the SN document, since it may apply to
>>>> other transports than just NETCONF.
>>>> /martin
>>>>> With this text, a RESTCONF publisher can support the feature dscp.
>>>>> And if someone attempts to use dscp with a NETCONF subscription to
>>>>> the same publisher, the error "dscp-unavailable" can be sent.
>>>>> Eric
>>>>> ...
>>>>>> /martin
>>> _______________________________________________
>>> netconf mailing list