Re: [CCAMP] Comparison of Flex-E YANG model parameters

<> Fri, 28 February 2020 07:42 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 597AE3A124A; Thu, 27 Feb 2020 23:42:07 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.797
X-Spam-Status: No, score=-1.797 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id YMDOHwu-V75l; Thu, 27 Feb 2020 23:42:02 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id D6AA63A1249; Thu, 27 Feb 2020 23:42:01 -0800 (PST)
Received: from (unknown []) by Forcepoint Email with ESMTPS id DC1375FE5E57BA3F6AB8; Fri, 28 Feb 2020 15:41:59 +0800 (CST)
Received: from (unknown []) by Forcepoint Email with ESMTPS id 5EE41A463F23946373EA; Fri, 28 Feb 2020 15:41:59 +0800 (CST)
Received: from ([]) by with SMTP id 01S7fUZi033498; Fri, 28 Feb 2020 15:41:30 +0800 (GMT-8) (envelope-from
Received: from mapi (kjyxapp03[null]) by mapi (Zmail) with MAPI id mid17; Fri, 28 Feb 2020 15:41:30 +0800 (CST)
Date: Fri, 28 Feb 2020 15:41:30 +0800 (CST)
X-Zmail-TransId: 2b055e58c42aa82c307d
X-Mailer: Zmail v1.0
Message-ID: <>
In-Reply-To: <>
Mime-Version: 1.0
From: <>
To: <>, <>, <>
Cc: <>, <>, <>
Content-Type: multipart/mixed; boundary="=====_001_next====="
X-MAIL: 01S7fUZi033498
Archived-At: <>
Subject: Re: [CCAMP] =?utf-8?q?Comparison_of_Flex-E_YANG_model_parameters?=
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussion list for the CCAMP working group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 28 Feb 2020 07:42:08 -0000

hi Rob, Daniele, Yuanlong,

Based on previous discussion, I think we are moving forward, although a little slow. I support to make more progress.

Please check the comments with [xb2] for more consideration.



Xiaobing NIU


发件人:Jiangyuanlong <>
收件人:Daniele Ceccarelli <>;Rob Wilton(rwilton) <>;牛小兵10019881019881;
抄送人 <>; <>; <>rg>;
日 期 :2020年02月28日 14:46
主 题 :RE: [CCAMP] Comparison of Flex-E YANG model parameters

Definitely I support this, and will be glad to work together with Rob and others who are interested on a common FlexE model.

I also think the initial model structure in the (
 is quite reasonable a start.





From: Daniele Ceccarelli []
Sent: Thursday, February 27, 2020 10:35 PM
To: Rob Wilton (rwilton) <>om>;
Subject: RE: [CCAMP] Comparison of Flex-E YANG model parameters


“If that is a reasonable approach, then do you think that it would be possible for both sets of authors to get together to produce a combined updated draft that takes into account the
 suggestions below and the prior proposed outline structure ( 
 Perhaps this could be done somewhere on github, with issues opened for the particular points that need further discussion?  I’m happy to be in those author discussions if it helps, but probably will have limited time actually working on a draft.”


This is exactly what I was suggesting…thank Rob for reiterate the message.






From: CCAMP <>On Behalf Of Rob Wilton (rwilton)
Sent: den 21 februari 2020 19:06
Subject: Re: [CCAMP] Comparison of Flex-E YANG model parameters


Hi Xiaobing,


Thanks for the comments.


I’ve given responses [RW] inline below, but please note that my aim here is just to come up with a common model that both sets of authors can agree as a starting point for further discussion. 
 I would suggest that the model should start relatively simple, but all the outstanding issues can and should be tracked, and the WG can discuss and decide whether or not the model should be extended to support each of those issues.


If that is a reasonable approach, then do you think that it would be possible for both sets of authors to get together to produce a combined updated draft that takes into account the
 suggestions below and the prior proposed outline structure ( 
 Perhaps this could be done somewhere on github, with issues opened for the particular points that need further discussion?  I’m happy to be in those author discussions if it helps, but probably will have limited time actually working on a draft.


Further comments inline.


From: <>
Sent: 21 February 2020 13:06
To: Rob Wilton (rwilton) <>
Subject: Re:Comparison of Flex-E YANG model parameters


Hi Rob,

I've added some comments with [xbn]. Please have a look.



Xiaobing Niu


发件人:RobWilton(rwilton) <>

 <>; <>; <>;



主题:RE: Re:Comparison of Flex-E YANG model parameters

Just pinging this thread to see if any of the authors had further thoughts on this.


I guess that the clock is ticking to get an updated model & draft before IETF 107, and it would be good to keep this work progressing.




Sent: 31 January 2020 16:40
To:; Rob Wilton (rwilton) <>
Subject: Re:Comparison of Flex-E YANG model parameters


Hi Rob, Yuanlong

Thanks for your detailed analysis and trade-off recommendation.

Recently, several contributions about FlexE have been discussed in ITU-T SG15 meeting and some more will be discussed in following days.  And more clear requirements about
 FlexE processing functions and related configuration are expected. I think that will be helpful to the model.



Xiaobing Niu




发件人:Jiangyuanlong <>

收件人:Rob Wilton (rwilton) <>; <>;
 <>; <>rg>;

抄送人:Yangfan (IP Standard) <>om>;


 Comparison of Flex-E YANG model parameters

Hi Rob,

Thanks a lot for the detailed comments, I agree with most of them (except several minor items, I will comment on them later).
I will be glad to work with you and the other authors on a common FlexE model draft.


-----Original Message-----
From: Rob Wilton (rwilton) [] 
Sent: Thursday, January 30, 2020 12:58 AM
Subject: Comparison of Flex-E YANG model parameters


Sorry for the delay.

After I think that we reached rough agreement on Flex-E model structure, I have now (with my limited knowledge) gone through the flex-e configuration parameters in draft-xiaobn and draft-jiang.  I think that I have hopefully given comments on all areas of the configuration.  I think that some aspects will be easier to resolve and reach conclusion on than other parts.

Hence, to try and make forward progress, I suggest that we try and defer getting stuck in discussion on the following topics:

  1.  Whether we need to have explicit configuration for both A and B calendars or not.  The model will definitely be simpler for client if this is not required, so I would recommend that we assume that it is not required, try and resolve the other points and then come back to this potentially thorny issue.

[xbn] If no explicit configuration for A/B calendar, some scenarios could not be supported.

  2.  Whether we need to have the "expected" configuration leaves.  I'm not sure I fully understand the intent of this, but I suspect that the intent is to use the calendar protocol as a way of checking that both sides are correctly configured.  My suggestion, to simplify the model, is to assume that in this case that the RX and TX calendar for client interfaces are the same, and hence this configuration could probably be expressed through a different "calendar-protocol-enable/tx-calendar-neg" setting.  But again, I suspect that this might be another can or worms that we might want to try and keep the lid on for the moment whilst we get agreement for the other parts of the model.  But, of course, this could be handled through a vendor or standard augmentations, or YANG features, but I still have a preference is trying to keep the base model as simple as we can.

 [xbn] Setting expected configuration info is aiming to strictly check the configuration in both ends. It can
 be relieved as you recommended above.

Okay, so I think that I understand that you are happy to defer this issue for the moment (perhaps track the issue on a github repro).

Barring these two points, you will see that I have tried to give a suggestion for the other configuration items, with an aim to try and get a compromise starting model that the authors and WG can move forward with.

Based on the comments that I receive, I propose updating the YANG model here (  If we do manage to find common ground over the next week or two, hopefully it will be possible for both sets of authors to take the combined model, polish it, and work together on a common FlexE model draft?

FlexE Group, Bonded-phy, instance and client configuration comparison:

(A) Group level configurable attributes:
    (A 1) Both drafts define a per group slot granularity (slot-granularity/cal-slot-gran)
     - both represented by an enum
     - both allow 5g and 25g (as per the OIF standard)
     - draft-jiang draft also allows "slot-others".
     - draft-jiang defines a default value (5g), but draft-xiaobn does not.
     - My recomendation:
       - The base model should only include 5g and 25g, because that is what is in the spec, and vendors can always deviate this leaf if required to support non-standard values.
       - I think that a default value is useful and should be defined, but possibly using 25g as the default because it is more forward looking (as bandwidths increase).
      [xbn] Maybe 5G slot is better, because 5G slot is more flexible to support various clients( e.g. BW lesss than 25G).

[RW] Sure, I think that this is a minor detail.

    (A 2) Both drafts support configuring Flex-E phy type (100g, 200g, 400g, 50g) (flexe-phy-type)
     - I'm not sure that this configuration is required, or just useful as an extra check.
     - Vendors could always deviate add "mandatory" if required.
     - I suggest that this leaf should be optional, and have no default value defined.

    (A 3) draft-xiaobn supports limiting the bandwidth for a flex-e group (flexe-gp-avb-bw)
     - This configuration seems to act as an additional constraint.  I don't see any harm in this configuration, but it seems somewhat redundant (given the bandwidth can be calculated for each member interface), and hence as a starting point, my preference would be to leave this out of a standard FlexE configuration data model.
     [xbn]When you take the Unequipped instance and Unavailable slot into consideration, the bandwidth can provide some sense. It gives the total BW directly.

[RW] Operationally reporting the available bandwidth definitely makes sense (which would take into account unavailable slots, etc).  But I’m less convinced
 of it as a configuration item.  E.g. what would it mean if the configured value doesn’t match the real bandwidth?  Would the config be rejected?

 [xb2] Could we make it readable only? When a slot is used or released, the available bandwidth changes accordingly. That can be used to jusge whether there is enough bandwdth available for a new requirement.

    (A 4) Both drafts seem to have a mechanism to control the calendar protocol (calendar-protocol-enable/tx-calendar-neg):
      - I would suggest changing the name of this leaf to "calendar-mode", and use an enumeration for future extensibility.
      - Enumeration choices could be
         "Static", - calendar configuration is static (and assumed to be the same for both RX and TX), calendar protocol information is ignored.
         "Verified" - calendar configuration is static (and assumed to be the same for both RX and TX), calendar protocol information is checked, otherwise the client interface is kept down in an errored state?  E.g. replacing the need for the "expected" configuration values in draft-xiaobn.

    [xbn] Originally this kind of ‘verifiy function’ was considered in ‘Static’ mode. Here you put it in a ‘Verified’ mode, and it is OK.

[RW] Okay.


         "Master-Slave" calendar configuration is static for TX, RX is determined from the calendar protocol (although perhaps should use different terminology for political correctness)
      - Having a default value for the calendar protocol makes sense, but I don't know what the appropriate default is.

    (A 5) draft-xiaobn defines reply-ca-mode:
      - I'm not quite sure what this leaf is for, and hence I question whether it is required since it appears to be somewhat implementation specific.

     [xbn] After recieving the CReq, the demux may delay for some time and then reply with the CA ,while not
 immediately reply. Some reasons for that are described in FlexE IAs. To consider both cases, the moment of replying CA should be flexible.

[RW] Okay, ideally it wouldn’t be necessary to expose this sort of detail into the model.  I would suggest that we leave this out to start off with, and
 raise a issue to track and discuss.

  [xb2] Could we leave this open that the default behaviour is replying CA immediately and other behaviour may be augmented later?

(B) Bonded-phy level configuration:
    (B 1) draft-xiaobn contains a "remote-phy-interface" as an interface-ref.  This configuration cannot work as an interface-ref (because it would attempt to validate against the local device's interface list).  I suspect that this isn't really configuration at all, but possibly just documentation.  As a starting point, my preference would be to leave this out of a standard FlexE configuration data model.  Vendors could always augment this in, with little interop issues.
    (B 2) Both drafts define mechanisms under the bonded-phy/[instance for draft-xiaobn] to specify the unavailable slots.  Rather than defining these as lists of unavailable slots, I would merge these into a single per bonded-phy leaf of ranges of what are the available slots, and if the leaf is not configured then all slots (for available instances) can be used.
        E.g. I am suggesting a per bonded-phy leaf:
            available-calender-slots = "1-15, 19, 20-29"

     [xbn] It’s
 ok. The slot number may be not consecutive.

[RW] Okay.

(C) Instance configuration:
    (C 1) draft-xiaobn has explicit support for flex-e "instances" from FlexE 2.1,(e.g. a 200G PHY is represented as 2x100G instances, a 400G PHY, has 4x100 instances).
     - I would think that any instance configuration should naturally be under the PHY, since from the spec, it seems to be phy specific, and relative to the phy.
     - FlexE has the ability to disable instances, but it must always be the highest instance numbers on a phy that are disabled.
        * One option is that this could be represented by a single configurable uint8 under the bonded-phy that indicates the number of supported-instances and defaults to all instances being supported if not configured.  [Note - this configuration is probably equivalent to the "uneq-flexe-instance", but in reverse sense and represented in a more concise way.]
        * An alternative option would be to infer which instances are disabled from the available-calendar-slot list.  E.g. for a 400G interfaces, stating available-slots="1-39" would imply that only instances 1 and 2 are in use, and instances 3 and 4 are disabled.
     - Otherwise I don't think that it is necessary to configure/define a separate calendar slot availability per instance.  I think that it will be more concise and simpler for clients if this is just configured under the bonded phy.  E.g., assuming 5G slots,  instance 0 would be slots 0-19, instance 1 would be slots 20-39, instance 2 would be slots 40-59, etc...

[xbn] OK

(D) "Expected configuration":
  (D 1) draft-xiaobn defines expected-group-number, expected-phy-map, expected-cal-cfg.
    - I don't think that these configuration items should be required.  These settings should either just be (i) regular configuration, i.e. covered by the other configuration items, or learned from the peer (e.g. if it can be carried via protocol).
    [xbn] It's related to the verifying function mentioned previously.

[RW] Okay, great.  Thanks for confirming that these are not required.


(E) "Calendar configuration"
  (E 1) draft-xiaobn defines tx-calendar, rx-calendar (and also tx-calendar-neg but that has been covered above).
   - I think that it is useful to be able to configure whether the A or B calendar is being used, defaulting to "A" if not specified.
   - I'm not convinced that the standard model should have rx-calendar. In the case that it is statically configured, I think that the standard model should assume that the RX and TX calendar are the same.

  [xbn] I think the unidirectional client service should also be considered.

[RW] Okay.  Can we initially start with the bidirectional only, and then raise an issue to discuss the unidirectional client service.  I think that it would be good to understand the real use cases here.

  [xb2] Ok. The bi-directional client service has more priorities compared with the uni-directional client service, and we suggest to keep track of the support of the uni-directional service.

(F) Client interface configuration
(F 1) From reading the OIF-FlexE spec, on the wire it looks like clients are bound to instances, but I don't think that this should be in the management model, it feels like it would be exposing unnecessary complexity.
   - draft-jiang allows each individual slot to be assigned to assigned to a phy and a slot on that phy.  This could end up being verbose.  E.g. 20 entries for a 100G client interface with 5G slot sizes.  Client ordering is explicitly specified by the client-slot-id leaf.
   - draft-xioabn maps to instance number and slot-id.  Client ordering isn't specified.  This would probably work if the list is "ordered-by user".
   - In both cases, I think that the configuration is a bit verbose, particularly for simple/normal configurations.  Hence, I would recommend a more compact format:
     - An ordered-by user list of phys, where for each phy the slots are specified in a range string, e.g. "1-10, 15-19".  The mapping from the client slot to phy slot is implicit from the client ordering of the phys and the slots.
     - This does effectively impose some restrictions (or simplifications):
       - The slots used by a client on a given phy are always in order, e.g.. I think that "10-1" or "6,3,2,9,5" should not be allowed.
       - You can't have some slots on one phy, then some on another phy, and then more on the first phy again.  All the slots on a given phy must be specified together (because the phy interface would be the list key).
     [xbn] It maybe work in this way after increasing or decreasing the width several times.

[RW] I still think that might be okay.  If the configuration is being done manually, then some of these changes could end up being service impacting, but
 if calendar negotiation was being used, then I think that this restriction should probably be okay (again, at least from a starting point)?

  [xb2] According to the restrictions above, the slot list configured for a client will be always ordered from lower number PHYs to higher number PHYs. That's OK.

 [xb2]There's a problem about the client and the client-interface under the FlexE group. we have talked about it.

   A client is Ethernent flow based on a MAC data rate. It can be generated within system or received from an Ethernet PHY. When configuring the client into a FlexE group, the client can be seen as an interface with MAC.

   In the proposed YANG model, the FlexE group includes PHYs, list of client-interface, and slots allocated for a client interface.  Because of the slots, the client-interface in Flexe group is diiferenet from the client that would be carried by the FlexE group, and they are two different kinds of interfaces. It's also one of the reason why Flexe-group is dfferent from LAG. They should be considered in different ways in the YANG model.

We think the model should include client, slot and PHY three parts. The client should be mapped into( or binded to) one of the client-interface in the FlexE group.

The client can be numbered with (Node ID+client number) in order to make it unique and conveniently maintained. While the client-interface can be numbered in the scope of a FlexE group( e.g. 1,2,3) or a node(Node ID+x). Even both can be numbered in the same value, they are diferent in essence.  

(F 2) Explicit configuration for calendar A vs calendar B.
  - draft-xioabn allows both A and B calendars to be specified separately, draft-jiang does not.
  - My view is that configuring both A and B calendars complicates the model, and I would prefer that it is left out of an initial joint WG draft, but I understand that significant further discussion on this point is probably required.
    [xbn]Some scenarios about configuring A/B to specific value are described in FlexE IAs. If this is not considered, the model would not be applicable for some applications.

[RW] Okay, but rather than explicitly configurating A and B calendars, it might be better to have a default calendar and an optional backup calendar, and
 then a separate config item to indicate which calendar is being used.  So, I agree that this should be discussed further, but was suggesting that we try and start from the simplest point and then add in the extra bits that are really required.

(F 3) rx calendar slots and expected calendar slots.
  - draft-xioabn allows, if the tx-calendar is in static mode, to specify the rx-calendar slots, and the expected A and B calendar slots.
  - I think that in static mode we should probably assume that RX calendar slots are the same as the TX calendar slots.
  - I don't think that the expected slots configuration should be required, or perhaps could be better modelled via the calendar protocol mode.
    [xbn] It’s related to two uni-directional flows with different BWs and directions. For
 example, in the same FlexE group, Flow A->Z uses slots[1, 2], while flow Z->A uses slots[2,3,4]. How to support this?

[RW] I think that for that more complex scenario we should rely on the calendar negotiation (for the standard model), but allow vendors to augment with
 this functionality if required.

  [xb2] It's a good idea to include a default calendar, optional backup calendar and configurable will-be-used calendar. That enables the network manager explicitly changes the calendar if he wants. 

(F 4) Bandwidth
   - draft-xiaobn defines client inteface bandwdth both in terms of the signal rate and mac-rate.
   - I think that having client interface bandwidth is required, and probably appropriate to define here under flexe client interface configuration.
   - Specifying the bandwidth in Gbps is probably sufficient, the signal rate can be inferred from the bandwidth.  Allowable values (10, 40, or m*25).
     [xbn] OK. 

(G) Operational values:
(G 1) draft-jiang defines flexe-phy-status:
   - I think that this is okay, although I question whether this would end up being a duplicate of information already available in a generic Ethernet Phy YANG model.
   - I propose that this is retained at this time.


Thanks, Rob

  [xb2]After more discussion, we think the flexe-phy-status is duplicated, and it's better not considered here.

(G 2) draft-jiang defines flexe-client-status:
   - I think that this is a duplicate of the status information that should be available under the clinet interface itself for all Ethernet interfaces