Re: [nvo3] 答复: Comments on NVO3 data plane requirements for OAM

["Deepak Kumar (dekumar)" <[e-mail]>]

Hi Mach,

Please see inline +++DK:


On 11/20/14 5:02 PM, "Mach Chen" <[e-mail]> wrote:

>Hi Tom,
>
>Please see my response inline...
>
>> -----Original Message-----
>> From: Tom Herbert [mailto:[e-mail]]
>> Sent: Friday, November 21, 2014 1:28 AM
>> To: Mach Chen
>> Cc: [e-mail]
>> Subject: Re: [nvo3] 答复: Comments on NVO3 data plane requirements for OAM
>> 
>> On Wed, Nov 19, 2014 at 5:54 PM, Mach Chen <[e-mail]> wrote:
>> > Hi Tissa,
>> >
>> > Thanks for your response!
>> >
>> > Please see my response inline...
>> >
>> >> -----Original Message-----
>> >> From: nvo3 [mailto:[e-mail]] On Behalf Of Tissa
>> >> Senevirathne
>> >> (tsenevir)
>> >> Sent: Wednesday, November 19, 2014 8:45 PM
>> >> To: Haoweiguo; Tom Herbert
>> >> Cc: Greg Mirsky; Tapraj Singh; Deepak Kumar (dekumar); [e-mail]
>> >> Subject: Re: [nvo3] 答复: Comments on NVO3 data plane requirements for
>> >> OAM
>> >>
>> >> Hi Weiguo, Mach et,al
>> >>
>> >> The discussion here is NVO3 data plane requirements for OAM. Like I
>> >> have said
>> >
>> > You are right, this discussion is about "NVO3 data plane requirements
>>for OAM",
>> but recently the focus is Performance Measurement (PM) requirement to
>>NVO3
>> that is also one of the OAM functions.
>> >
>> >> earlier,  we do not need to complicate the Data Plane. Can you
>> >> explain to me
>> >
>> > "Complicate/simple" is not the goal, the goal is to define a
>>reasonable solution
>> that can satisfy the requirement. That's why I agree with Greg that we
>>should
>> firstly make the agreement on the requirement.
>> >
>> Mach,
>> 
>> The nvo3 OAM requirements draft
>> (draft-ashwood-nvo3-oam-requirements-01) seems to already contain a
>>fairly
>> comprehensive list of requirements. Particularly apropos to this
>>discussion are:
>> 
>>    R13) NVO3 OAM frames MUST be forwarded along the same path (i.e.,
>>    links (including LAG members) and nodes) as the NVO3 data frames.
>> 
>>    R16) NVO3 OAM should be extensible such that new functionality and
>>    information elements related to this functionality can be introduced
>>    in the future.
>> 
>> I believe that an an active OAM message format like Tissa describes
>>would meet
>> these and most of the others in that draft.
>
>There are also the following requirements listed in the draft:
>
>   R7) NVO3 OAM MUST support measurement of per VNI frame loss between
>   two NV Edge devices that support the same VNI within a given NVO3
>   domain.
>
>   R8) NVO3 OAM MUST support measurement of per VNI two-way frame delay
>   between two NV edge devices that support the same VNI within a given
>   NVO3 domain.
>
>   R9) NVO3 OAM MUST support measurement of per VNI one-way frame delay
>   between two NV Edge devices that support the same VNI within a given
>   NVO3 domain.
>
>   R10) NVO3 OAM MUST support measurement of per VNI frame delay
>   variation between two NV Edge devices that support the same VNI
>   within a given NVO3 domain.
>
>
>> 
>> If a passive mechanism is indeed required,
>
>I personally think it is indeed required, and we also received such
>requirements from the operators.
>
>> then we need to consider how to
>> meet the extensibility requirement. I don't believe that allocating two
>>bit flags in
>> the encapsulation header is at all an extensible solution. The reserved
>>header
>> bits are too a precious resource to be allocated for such a narrow
>>purpose and for
>
>Looking through the bits in some headers, we could find that every bit in
>a header has its own purpose. It's better that one bit could be defined
>for as many usages as possible, but there is always tradeoff.
>
>As for the two bits for passive PM that include loss, one/two way delay,
>delay variation and throughput, I may not think this is a narrow purpose.
>And if you want, you may use the marking bit for some policies control.
>
>> something not required for protocol operation. As previously discussed
>>in this
>> thread, using one bit to get one-way time delay measurements is not
>>even viable
>> in a lot deployments-- in this case we probably need timestamps to get
>>RTT.
>
>For the deployments in question, the challenge for one way time delay is
>time synchronization and its accuracy. The solution of using one-bit for
>one-way delay is really workable, there have been some prototypes and
>experiments show that.
>
>Since time synchronization is not needed for RTT, IMHO, measure RTT
>should be the easiest way to go.
> 
>
>> 
>> To support passive OAM support in GUE, I would probably propose to add a
>> generic optional "measurement" field. This would provide some number of
>>bits
>> in the header that can be used for passive measurement (possibly a few
>> different sizes say 32, 64, 128 bits). The field can be structured to
>>allow different
>> mechanisms (e.g. include timestamps for RTT measurement). This also
>>reduces
>> the constraints on the measurement techniques, for instance the marking
>> technique might no longer limited to use a single bit which should
>>reduce the
>> complexity needed to deal with OOO or packet loss.
>
>Even with the solution as above, seems there needs at least one bit(at
>the fix position of the header) that indicates there is an optional field
>exist. In the case we could have opportunity and "enough" reserved bits
>to allocate for the marking bits, I'd like to suggest allocating two bits
>for passive PM. 
>
>In addition, I see the value of you proposed optional "measurement"
>field, it could be used to carry some correlation (e.g., block/period
>number) and timestamp information, then combine with the marking bit, it
>can greatly simplify the marking based solution.

+++DK:
I think adding information regarding measurement field, block, period,
etc. is not required in data path as more information reduces the mtu, and
this can easily be added if required by TLV to OAM functionality with new
subtype (as this is control or configuration functionality).
Also even passive oam loss measurement solution to calculating loss is not
accurate as packets can arrive late outside the measuring blocks. Even in
that case to get accurate measurement instead of ipfix method, better to
use OAM to exchange these marked packet counters on both ends and do loss
measurement between two consecutive loss measurement replies.

For loss measurement, why we have to count traffic for marked packets only
and not maintain counters per flow?

Thanks,
Deepak
>
>Thanks,
>Mach
>
>> 
>> Tom
>> 
>> >
>> >> what difference it make to the data plane whether it is
>> >> active/passive or some other means of OAM.
>> >
>> > Active/passive is mainly regarding to PM which normally includes
>>Active and
>> Passive PM.
>> >
>> > Active PM measures the injected packets (e.g., OAM packets) to
>>evaluate the
>> performance of a path. Passive PM measures the performance of the
>>real/live
>> traffic of a path, it reflects the real performance of the path. For
>>more detail
>> about active/passive PM, you may refer to the material of IPPM WG.
>> >
>> >>
>> >> All what it needs to know is that the packet is an OAM packet and it
>> >> is addressed to the local device,
>> >
>> > What you are talking are just part of the OAM functions (e.g., CC,
>>CV), for
>> passive PM, OAM packets may not be needed.
>> >
>> >
>> > Best regards,
>> > Mach
>> >
>> >>
>> >> -----Original Message-----
>> >> From: Haoweiguo [mailto:[e-mail]]
>> >> Sent: Tuesday, November 18, 2014 7:50 PM
>> >> To: Tissa Senevirathne (tsenevir); Tom Herbert
>> >> Cc: Greg Mirsky; Tapraj Singh; Deepak Kumar (dekumar); [e-mail]
>> >> Subject: RE: [nvo3] 答复: Comments on NVO3 data plane requirements for
>> >> OAM
>> >>
>> >> Hi  Tissa,
>> >> Your solution is active OAM,  i think it is a basic and important
>> >> solution in whole OAM framework.The disccussed thread is about
>>passive
>> OAM.
>> >>  Both active and passive OAM have its pros/cons, both have its
>> >> usecases and scenarios.The regular method for passive OAM is to add
>> >> marking bits in packet header, in NVO3 case, the marking bits had
>> >> better be set in NVO3 header.But just as Greg said,currently it's
>> >> unfortunate that there is no accepted OAM requirements, gap analysis,
>> >> and etc in the WG. We hope this work could be progressed more
>>quickly.
>> >> Thanks
>> >> weiguo
>> >> ________________________________________
>> >> From: Tissa Senevirathne (tsenevir) [[e-mail]]
>> >> Sent: Wednesday, November 19, 2014 8:25
>> >> To: Tom Herbert
>> >> Cc: Greg Mirsky; Tapraj Singh; Deepak Kumar (dekumar); [e-mail]
>> >> Subject: Re: [nvo3] 答复: Comments on NVO3 data plane requirements for
>> >> OAM
>> >>
>> >> Hi Tom
>> >>
>> >> Your interpretation is correct. The entropy allows OAM packets to
>> >> follow the same path as the data packet.
>> >>
>> >> As I noted earlier on in the thread, OAM processing would not kick in
>> >> unless address matches the MEP/MIP. If address match MEP/MIP and OAM
>> >> bit is set, then OAM processing begins.
>> >>
>> >> -----Original Message-----
>> >> From: Tom Herbert [mailto:[e-mail]]
>> >> Sent: Tuesday, November 18, 2014 4:20 PM
>> >> To: Tissa Senevirathne (tsenevir)
>> >> Cc: Greg Mirsky; Tapraj Singh; [e-mail]; Deepak Kumar (dekumar)
>> >> Subject: Re: [nvo3] 答复: Comments on NVO3 data plane requirements for
>> >> OAM
>> >>
>> >> On Tue, Nov 18, 2014 at 3:54 PM, Tissa Senevirathne (tsenevir)
>> >> <[e-mail]> wrote:
>> >> > Greg
>> >> >
>> >> >
>> >> >
>> >> > I disagree with you on FM and PM cannot be achieved in ECMP
>> environment.
>> >> > Significant amount of work has gone in to this area during TRILL
>>OAM.
>> >> > Please check the use of Flow entropy functionality proposed in
>>NVO3 OAM.
>> >> >
>> >> >
>> >> >
>> >> > https://tools.ietf.org/html/draft-tissa-nvo3-oam-fm-00
>> >> >
>> >> Tissa,
>> >>
>> >> If I am reading this correctly, the OAM message would be composed of
>> >> the encapsulation header, followed by 128 bytes of which contains a
>> >> pseudo header for switching, followed by a self defining OAM message.
>> >> The OAM bit is only used at the receiver to distinguish data messages
>> >> for OAM messages for processing. Is this interpretation correct?
>> >>
>> >> Thanks,
>> >> Tom
>> >>
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > From: nvo3 [mailto:[e-mail]] On Behalf Of Greg Mirsky
>> >> > Sent: Tuesday, November 18, 2014 3:03 PM
>> >> > To: Tapraj Singh
>> >> > Cc: [e-mail]
>> >> > Subject: Re: [nvo3] 答复: Comments on NVO3 data plane requirements
>> >> > for OAM
>> >> >
>> >> >
>> >> >
>> >> > Hi Tapraj,
>> >> >
>> >> > though I agree and support with idea of having OAM flag in NVO3
>> >> > header I have to point to:
>> >> >
>> >> > absence of WG agreed upon OAM Requirements; no gap analysis of
>> >> > tools for NVO3 OAM; OAM flag does not help passive performance
>> >> > measurement marking method (two bit-long field for marking in fixed
>> position).
>> >> >
>> >> > I agree that PW VCCV and GAL/G-ACh can be viewed as MPLS
>> >> > identification of OAM packet (though not necessarily OAM). But IP
>> >> > clearly doesn't have such identification for OAM and that, in part,
>> >> > why in-band requirement for IP OAM, both FM and Active PM, is not
>> >> > attainable
>> >> (ECMP environment).
>> >> >
>> >> > Regards,
>> >> >
>> >> > Greg
>> >> >
>> >> >
>> >> >
>> >> > On Tue, Nov 18, 2014 at 1:31 PM, Tapraj Singh <[e-mail]>
>>wrote:
>> >> >
>> >> > Hi All,
>> >> >
>> >> >  I totally agree with the point made by Deepak and Tissa here.
>> >> > Our OAM should follow the data path for services as much as
>> >> > possible and all other protocol specific information should be in
>> >> > the OAM protocol specific TLVs.
>> >> >
>> >> > LAYER2 OAM
>> >> >
>> >> > In term of identify the OAM packet, first level of identification
>> >> > for
>> >> > L2 OAM Should be the MAC address and send level of hierarchy should
>> >> > be the ether type or OUI.
>> >> > No other OAM Specific field should be allowed in the packet header.
>> >> >
>> >> >  Please note that L3 OAM and MPLS also follow the same principle.
>> >> >
>> >> > Thanks
>> >> > Tapraj
>> >> >
>> >> >
>> >> > On 11/17/14 12:39 PM, "Deepak Kumar (dekumar)" <[e-mail]>
>> >> wrote:
>> >> >
>> >> >>I Agree with Tissa below. My Goal also was to point out that
>> >> >>instead of complicating the header, we can do OAM performance
>> >> >>within OAM channel itself and this is extensible and can be done in
>> >> >>hardware which is why mostly things are added in header.
>> >> >>
>> >> >>Also, Operators keep asking for new OAM tools (Fault detection,
>> >> >>verification, isolation, Interworking, alarm, putting service in
>> >> >>maintenance and perform test)  and Performance tools, eg:
>> >> >>(Delay/Jitter, Actual Loss Measurement, Synthetic Loss, loopback
>> >> >>signaling like TDM, Generate frames to verify qos etc.) and so OAM
>> >> >>Channel solution will be extensible.
>> >> >>
>> >> >>Thanks,
>> >> >>Deepak
>> >> >>
>> >> >>On 11/17/14 8:47 AM, "Tissa Senevirathne (tsenevir)"
>> >> >><[e-mail]>
>> >> >>wrote:
>> >> >>
>> >> >>>I think we are complicating OAM beyond what it is needed.
>> >> >>>
>> >> >>>As far as packet encapsulation is concern, all what is needed is
>> >> >>>single bit. This bit is needed to prevent OAM packets leaking out
>> >> >>>from the domain.
>> >> >>>
>> >> >>>Termination of OAM and processing of it happen based on the
>> >> >>>addressing in the packet.
>> >> >>>
>> >> >>>E.g. if Address matches and OAM bit is set then it is an OAM
>> >> >>>packet addressed to the local MEP/MP.
>> >> >>>
>> >> >>>Not other way around. Why? Because we want OAM to be as closely as
>> >> >>>possible follow the Data path.
>> >> >>>
>> >> >>>If we need to have performance and delay measurements, we SHOULD
>> >> >>>NOT mutate the packet header.
>> >> >>>
>> >> >>>Instead OAM specific extensions should be in the OAM shim.
>> >> >>>
>> >> >>>As an example. You could have packet fragment (which is sometimes
>> >> >>>called flow entropy) and at the end of that you can have all of
>> >> >>>the stuff you need in the world of OAM.
>> >> >>>
>> >> >>>Hope this clarify
>> >> >>>
>> >> >>>Thanks
>> >> >>>Tissa
>> >> >>>-----Original Message-----
>> >> >>>From: nvo3 [mailto:[e-mail]] On Behalf Of Tom Herbert
>> >> >>>Sent: Monday, November 17, 2014 8:02 AM
>> >> >>>To: Marc Binderberger
>> >> >>>Cc: Greg Mirsky; Mach Chen; Deepak Kumar (dekumar); [e-mail];
>> >> >>>Haoweiguo; Larry Kreeger (kreeger); Vero Zheng; Jon Hudson
>> >> >>>Subject: Re: [nvo3] 答复: Comments on NVO3 data plane requirements
>> >> >>>for OAM
>> >> >>>
>> >> >>>On Mon, Nov 17, 2014 at 12:01 AM, Marc Binderberger
>> >> >>><[e-mail]>
>> >> >>>wrote:
>> >> >>>> Hello Deepak et al.,
>> >> >>>>
>> >> >>>> so this sounds like we need more than just a (2nd) bit for delay
>> >> >>>>measurement.
>> >> >>>> Seems we need an optional header extension or a TLV to carry all
>> >> >>>>the  information (timestamps, oam Subtype). Sounds definitely
>> >> >>>>more than a  32/64bit header could carry (*).
>> >> >>>>
>> >> >>>> The optional header extension, when done similar to GUE, has a
>> >> >>>> fixed position. For the TLV this would be an additional
>> >> >>>> requirement. This would allow for hardware-stamping.
>> >> >>>>
>> >> >>>The alternative is to do active delay measurement using
>>request/reply.
>> >> >>>We should be able to define the requirements so that an OAM
>> >> >>>message corresponding to a flow which would be routed in exactly
>> >> >>>the same way as a data message for the flow. Larry mentioned that
>> >> >>>we might even want to put a "fake" packet header as the first part
>> >> >>>of the encapsulated payload of an OAM message for instance.
>> >> >>>
>> >> >>>> Now if we introduce such an OAM extension header it could as
>> >> >>>> well carry the "first" bit we discussed for packet loss
>>measurement (?).
>> >> >>>>
>> >> >>>>
>> >> >>>> Regards, Marc
>> >> >>>>
>> >> >>>> (*: at least all proposals so far have a base header that fits
>> >> >>>> into
>> >> >>>> 32/64 bit, plus IP and potential UDP)
>> >> >>>>
>> >> >>>>
>> >> >>>>
>> >> >>>>
>> >> >>>> On Sun, 16 Nov 2014 16:44:54 +0000, Deepak Kumar (dekumar)
>>wrote:
>> >> >>>>> Hi,
>> >> >>>>>
>> >> >>>>> Please see inline +++DK:
>> >> >>>>>
>> >> >>>>> On 11/14/14 11:09 AM, "Jon Hudson" <[e-mail]>
>>wrote:
>> >> >>>>>
>> >> >>>>>>
>> >> >>>>>> One comment in line....
>> >> >>>>>>
>> >> >>>>>>> On Nov 13, 2014, at 11:47 PM, Vero Zheng
>> >> >>>>>>><[e-mail]>
>> >> >>>>>>>wrote:
>> >> >>>>>>>
>> >> >>>>>>> Hi Tom,
>> >> >>>>>>>
>> >> >>>>>>> Please see in-line.
>> >> >>>>>>>
>> >> >>>>>>> BR, Vero
>> >> >>>>>>>
>> >> >>>>>>>> -----Original Message-----
>> >> >>>>>>>> From: nvo3 [mailto:[e-mail]] On Behalf Of Tom
>> >> >>>>>>>> Herbert
>> >> >>>>>>>> Sent: Friday, November 14, 2014 4:27 PM
>> >> >>>>>>>> To: Mach Chen
>> >> >>>>>>>> Cc: Greg Mirsky; Haoweiguo; Marc Binderberger; Larry
>> >> >>>>>>>> Kreeger; [e-mail]
>> >> >>>>>>>> Subject: Re: [nvo3] 答复: Comments on NVO3 data plane
>> >> >>>>>>>> requirements for OAM
>> >> >>>>>>>>
>> >> >>>>>>>> On Wed, Nov 12, 2014 at 5:13 PM, Mach Chen
>> >> >>>>>>>> <[e-mail]>
>> >> >>>>>>>> wrote:
>> >> >>>>>>>>> Hi Tom,
>> >> >>>>>>>>>
>> >> >>>>>>>>>> -----Original Message-----
>> >> >>>>>>>>>> From: Tom Herbert [mailto:[e-mail]]
>> >> >>>>>>>>>> Sent: Thursday, November 13, 2014 3:11 AM
>> >> >>>>>>>>>> To: Marc Binderberger
>> >> >>>>>>>>>> Cc: Mach Chen; Greg Mirsky; Haoweiguo; [e-mail];
>> >> >>>>>>>>>> Larry Kreeger
>> >> >>>>>>>>>> Subject: Re: [nvo3] 答复: Comments on NVO3 data plane
>> >> >>>>>>>>>> requirements for OAM
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> On Wed, Nov 12, 2014 at 2:11 AM, Marc Binderberger
>> >> >>>>>>>>>> <[e-mail]>
>> >> >>>>>>>> wrote:
>> >> >>>>>>>>>>> Hello Mach,
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> so for delay measurement you use the color flag to mark a
>> >> >>>>>>>>>>> single packet, which helps the receiver to pick the right
>> >> >>>>>>>>>>> packet?  And repeat this every time period T ?
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>    ...000100000010000001000...
>> >> >>>>>>>>>> Is there there a draft or description of how this
>> >> >>>>>>>>>> algorithm would work? Seems like there would need to be
>> >> >>>>>>>>>> quite a bot of synchronization needed between end points
>> >> >>>>>>>>>> (synchronized clocks, provisions to correlate measurements
>> >> >>>>>>>>>> correctly with lost packets, replicated packets, etc.).
>> >> >>>>>>>>>> Also, what is envisioned for range for the period?
>> >> >>>>>>>>>
>> >> >>>>>>>>> Here is a reference
>> >> >>>>>>>>
>> >> >>>>>>>> https://datatracker.ietf.org/doc/draft-chen-ippm-coloring-ba
>> >> >>>>>>>> sed
>> >> >>>>>>>> -ip
>> >> >>>>>>>> fpm-fr
>> >> >>>>>>>> amew
>> >> >>>>>>>> ork/.
>> >> >>>>>>>>
>> >> >>>>>>>> Thanks for the pointer. Regarding the need for synchronized
>> >> >>>>>>>>clocks  to measure delay, I consulted our local NTP expert.
>> >> >>>>>>>>The host clock  jitter we currently see in our network is
>> >> >>>>>>>>currently usually  greater than one-way packet delay (in some
>> >> >>>>>>>>cases much greater), so  in his words:
>> >> >>>>>>>> "measuring one-way packet delays using host clocks is a lost
>> >> >>>>>>>>cause".
>> >> >>>>>>>> Please take this as just one data point!
>> >> >>>>>>
>> >> >>>>>> <Jon> Thank you. As someone who has managed NTP more times
>> and
>> >> >>>>>>for  more years than I care to admit, this is a very good
>> >> >>>>>>datapoint to consider.
>> >> >>>>>> NTP helps many understand that time is relative.
>> >> >>>>>
>> >> >>>>> +++DK: As per our experience in carrier Ethernet we supported
>> >> >>>>> +++one way
>> >> >>>>> delay and never found NTP useful even for our lab networks (I
>> >> >>>>>am referring software based NTP NTPv3).
>> >> >>>>> As mentioned below IEEE 1588v2 will vary based on equipment and
>> >> >>>>>operator networks but in our testing we found it very precise if
>> >> >>>>>properly deployed.
>> >> >>>>> IEEE 1588v2 is very precise if phy based timestamping is used.
>> >> >>>>>Even  timestamping at NP level provided great results for one
>>way
>> delay.
>> >> >>>>>
>> >> >>>>> If we want to accurately measure two way delay we need 4
>> >> >>>>> timestamp total on receiver of frame (this is to avoid
>> >> >>>>> processing time that's taken for reply by software as hardware
>> >> >>>>> can put timestamp at lower layer without doing delay and jitter
>> calculation).
>> >> >>>>> For one way delay we will require 2 timestamp, so lower layer
>> >> >>>>> hardware can timestamp before packet is punted to software.
>> >> >>>>>
>> >> >>>>> As mentioned below I agree 8 byte IEEE 1588 timestamp is
>>required.
>> >> >>>>>
>> >> >>>>> We should also look for Synthetic OAM applicability for
>> >> >>>>>performance ('O'
>> >> >>>>> bit can be overloaded to do both Fault and performance if OAM
>> >> >>>>>is defined with different oam Subtype for Delay and Loss frames
>> >> >>>>>and it will not be too deep hardware inspection) as that give
>> >> >>>>>large flexibility (synthetic/real loss measurement,
>> >> >>>>>Availability/unavailability, on-demand and pro-active
>> >> >>>>>performance) and can be run on all flows of ECMP.
>> >> >>>>>
>> >> >>>>> Thanks,
>> >> >>>>> Deepak
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>>
>> >> >>>>>>> [Vero] Thanks for this. What about the current experience
>> >> >>>>>>> with
>> >> >>>>>>> 1588v2 then?
>> >> >>>>>>>>
>> >> >>>>>>>>> Yes, it does need some synchronization. As for the range,
>> >> >>>>>>>>> it depends on two
>> >> >>>>>>>> factors, one is the implementation limitation, the other the
>> >> >>>>>>>> requirement of the operators. In the above reference, the
>> >> >>>>>>>> suggested periods are 1s, 10s, 1min, 10min and 1h.
>> >> >>>>>>>> I think if we were implementing delay measurement in GUE, I
>> >> >>>>>>>> would advocate add a 64 bit optional field for timestamp,
>> >> >>>>>>>> probably containing source time stamp, and echoed timestamp
>> >> >>>>>>>> for a flow (usec resolution and similar in design TCP
>> >> >>>>>>>> timestamp option). This easily gives a precise RTT, and if
>> >> >>>>>>>> clocks are precisely synchronized then one way latency
>>could be
>> calculated also.
>> >> >>>>>>> [Vero] If the source timestamp could be carried, it could
>> >> >>>>>>> also be used for packet loss calculation/correlation.
>> >> >>>>>>>
>> >> >>>>>>>> Thanks,
>> >> >>>>>>>> Tom
>> >> >>>>>>>>
>> >> >>>>>>>>> Best regards,
>> >> >>>>>>>>> Mach
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> Thanks,
>> >> >>>>>>>>>> Tom
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> One question I still have is: why is the measurement done
>> >> >>>>>>>>>>> in the NVE
>> >> >>>>>>>> header?
>> >> >>>>>>>>>>> The outer header is IP/IPv6, so couldn't we use the
>> >> >>>>>>>>>>> coloring for the
>> >> >>>>>>>>>>> IP/IPv6 header, assuming this is defined?
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> Thanks & Regards,
>> >> >>>>>>>>>>> Marc
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>> On Wed, 12 Nov 2014 09:34:52 +0000, Mach Chen wrote:
>> >> >>>>>>>>>>>> Hi Tom,
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>> -----Original Message-----
>> >> >>>>>>>>>>>>> From: Tom Herbert [mailto:[e-mail]]
>> >> >>>>>>>>>>>>> Sent: Wednesday, November 12, 2014 5:06 PM
>> >> >>>>>>>>>>>>> To: Mach Chen
>> >> >>>>>>>>>>>>> Cc: Greg Mirsky; Haoweiguo; [e-mail]; Larry
>> >> >>>>>>>>>>>>> Kreeger
>> >> >>>>>>>>>>>>> (kreeger)
>> >> >>>>>>>>>>>>> Subject: Re: [nvo3] 答复: Comments on NVO3 data plane
>> >> >>>>>>>>>>>>> requirements for OAM
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> On Wed, Nov 12, 2014 at 12:55 AM, Mach Chen
>> >> >>>>>>>>>>>>> <[e-mail]>
>> >> >>>>>>>>>>>>> wrote:
>> >> >>>>>>>>>>>>>> Hi Greg and all,
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Single bit is not sufficient if someone wants to
>> >> >>>>>>>>>>>>>>perform loss and delay measurement  simultaneously,
>> >> >>>>>>>>>>>>>>then two bits needed.
>> >> >>>>>>>>>>>>> Is that necessary? Can they share the same time quantum
>> >> >>>>>>>>>>>>>(as  well as other metrics maybe to be added later)? In
>> >> >>>>>>>>>>>>>all the  protocols mentioned, the reserved bits are a
>> >> >>>>>>>>>>>>>somewhat precious resource.
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> Yes, it's necessary if there is ECMP.
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> Given one bit is used for both loss and delay
>> >> >>>>>>>>>>>> measurement, for loss measurement, it periodically set
>> >> >>>>>>>>>>>> and clear the marking bit, a flow is divided into
>> >> >>>>>>>>>>>> consecutive blocks, and then the counting and
>>calculating are
>> based on each block.
>> >> >>>>>>>>>>>> This is fine for loss measurement.
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> For delay measurement, it has to make sure the
>> >> >>>>>>>>>>>> timestamps (collected at sender and receiver) are for
>>the
>> same packet.
>> >> >>>>>>>>>>>> Presumably, the time when changing the marking bit is
>> >> >>>>>>>>>>>> right time to get
>> >> >>>>>>>> the timestamps.
>> >> >>>>>>>>>>>> Since there is ECMP, the first packet of a block at the
>> >> >>>>>>>>>>>>sender  may probably different from the first packet at
>> >> >>>>>>>>>>>>the receiver,  thus it will get the mismatched timestamps
>> >> >>>>>>>>>>>>to calculate the delay.
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> Best regards,
>> >> >>>>>>>>>>>> Mach
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> Tom
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Best regards,
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Mach
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> From: nvo3 [mailto:[e-mail]] On Behalf Of
>> >> >>>>>>>>>>>>>> Greg Mirsky
>> >> >>>>>>>>>>>>>> Sent: Wednesday, November 12, 2014 8:05 AM
>> >> >>>>>>>>>>>>>> To: Haoweiguo
>> >> >>>>>>>>>>>>>> Cc: [e-mail]; Larry Kreeger (kreeger)
>> >> >>>>>>>>>>>>>> Subject: Re: [nvo3] 答复: Comments on NVO3 data plane
>> >> >>>>>>>> requirements
>> >> >>>>>>>>>>>>>> for OAM
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Dear All,
>> >> >>>>>>>>>>>>>> agree with Weiguo, single bit flag in fixed position
>> >> >>>>>>>>>>>>>> would be sufficient and HW-friendly.
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Regards,
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Greg
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> On Tue, Nov 11, 2014 at 3:51 PM, Haoweiguo
>> >> >>>>>>>>>>>>>> <[e-mail]>
>> >> >>>>>>>>>>>>> wrote:
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Hi Larry,
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> For marking purpose, i think one bit maybe OK, fixed
>> >> >>>>>>>>>>>>>> fields in
>> >> >>>>>>>>>>>>>> NVO3 header is precious. I would like it is set in
>> >> >>>>>>>>>>>>>> fixed field, rather than in option field. Because
>> >> >>>>>>>>>>>>>> chipset normally can't process optional field, it is
>> >> >>>>>>>>>>>>>> hard to realize in-band performance measurement if
>> >> >>>>>>>>>>>>>> using optional
>> >> >>>>>>>>>>>>> field for marking.
>> >> >>>>>>>>>>>>>> For other real time congestion control function, maybe
>> >> >>>>>>>>>>>>>> more bits are needed.
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Thanks
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> weiguo
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> ________________________________
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> 发件人: Larry Kreeger (kreeger) [[e-mail]]
>> >> >>>>>>>>>>>>>> 发送时间: 2014年11月12日 4:33
>> >> >>>>>>>>>>>>>> 收件人: Haoweiguo; Greg Mirsky
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> 抄送: [e-mail]
>> >> >>>>>>>>>>>>>> 主题: Re: [nvo3] Comments on NVO3 data plane
>> >> requirements
>> >> >>>>>>>>>>>>>> for
>> >> >>>>>>>> OAM
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Hi Weiguo,
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> What do you envision this marking looking like?  e.g.
>> >> >>>>>>>>>>>>>> is it just a single flag bit, or large field with a
>> >> >>>>>>>>>>>>>> counter or sequence number, or some kind of flow ID?
>> >> >>>>>>>>>>>>>> If not a single flag, how large do you see the field
>> >> >>>>>>>>>>>>> being?
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> If it is more than a flag (and I assume it would be),
>> >> >>>>>>>>>>>>>> and is not mandatory for all implementations, then it
>> >> >>>>>>>>>>>>>> seems to fall into the category of optional
>>extensions.
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Thanks, Larry
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> From: Haoweiguo <[e-mail]>
>> >> >>>>>>>>>>>>>> Date: Tuesday, November 11, 2014 10:18 AM
>> >> >>>>>>>>>>>>>> To: Greg Mirsky <[e-mail]>
>> >> >>>>>>>>>>>>>> Cc: "[e-mail]" <[e-mail]>
>> >> >>>>>>>>>>>>>> Subject: [nvo3] 答复: Comments on NVO3 data plane
>> >> >>>>>>>>>>>>>> requirements
>> >> >>>>>>>> for
>> >> >>>>>>>>>>>>>> OAM
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Hi Greg,
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> I fully agree with you.
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> The real time OAM is passive performance measurement
>> >> >>>>>>>>>>>>>> methods. I would like
>> >> >>>>>>>>>>>>>> NVO3 data encapsulation has a field for marking and
>> >> >>>>>>>>>>>>>> not affect forwarding of packets, the marking field is
>> >> >>>>>>>>>>>>>> only used for performance measurement. The
>> >> >>>>>>>>>>>>>> NVO3 packet with this marking flag don't need to be
>> >> >>>>>>>>>>>>>> sent to control plane, it is different from
>> >> >>>>>>>>>>>>>> OAM(ping/Trace) packet processing.
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Thanks
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> weiguo
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> ________________________________
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> 发件人: Greg Mirsky [[e-mail]]
>> >> >>>>>>>>>>>>>> 发送时间: 2014年11月12日 4:07
>> >> >>>>>>>>>>>>>> 收件人: Haoweiguo
>> >> >>>>>>>>>>>>>> 抄送: [e-mail]
>> >> >>>>>>>>>>>>>> 主题: Re: [nvo3] Comments on NVO3 data plane
>> >> requirements
>> >> >>>>>>>>>>>>>> for
>> >> >>>>>>>> OAM
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Hi Weiguo,
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> marking groups of packets that belong to the
>> >> >>>>>>>>>>>>>> particular flow to facilitate measurement of some
>> >> >>>>>>>>>>>>>> performance metric, whether loss or delay/delay
>> >> >>>>>>>>>>>>>> variation, may be viewed as one of passive performance
>> >> >>>>>>>>>>>>> measurement methods.
>> >> >>>>>>>>>>>>>> But such marking should not alter, at least not
>> >> >>>>>>>>>>>>>> significantly alter, treatment of data flow in the
>>network.
>> >> >>>>>>>>>>>>>> Because of that, I believe, OAM flag should not be
>> >> >>>>>>>>>>>>>> used for marking as that will force punting marked
>> >> >>>>>>>>>>>>>> packets from fast forwarding path to the control
>> >> >>>>>>>>>>>>>> plane. But it might be good to have a field in NVO3
>> >> >>>>>>>>>>>>>> header that may be used for marking and not affect
>> >> >>>>>>>>>>>>>> forwarding of
>> >> >>>>>>>> packets if altered.
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Regards,
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Greg
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> On Tue, Nov 11, 2014 at 12:34 AM, Haoweiguo
>> >> >>>>>>>>>>>>>> <[e-mail]>
>> >> >>>>>>>>>>>>> wrote:
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Hi All,
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> I maybe not clearly said in today’s NVO3 meeting, pls
>> >> >>>>>>>>>>>>>>allow  me to reiterate the OAM data plane requirements
>> >> >>>>>>>>>>>>>>on the mail list.
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Currently NVO3 data plane encapsulation only includes
>> >> >>>>>>>>>>>>>>one OAM flag, it is used for Ping/Trace similar
>>applications.
>> >> >>>>>>>>>>>>>> This kind of OAM application is initiated by operators
>> >> >>>>>>>>>>>>>>for  network connectivity verification, normally when
>> >> >>>>>>>>>>>>>>network failure occurs.
>> >> >>>>>>>>>>>>>> There is another OAM requirements of real time OAM or
>> >> >>>>>>>>>>>>>>synthesizing OAM. It can be used for
>> >> >>>>>>>>>>>>> packet loss detection in real time.
>> >> >>>>>>>>>>>>>> When ingress NVE receives traffic from local TS, it
>> >> >>>>>>>>>>>>>>gets packet statistics, and mark(coloring) the OAM flag
>> >> >>>>>>>>>>>>>>relying on local policy when it performs
>> >> >>>>>>>>>>>>>> NVO3 encapsulation. When egress NVEs receives the
>> >> >>>>>>>>>>>>>>traffic,  it decapsulates
>> >> >>>>>>>>>>>>>> NVO3 encapsulation, and gets packet statistics with
>> >> >>>>>>>>>>>>>>the real  time OAM flag marking. By comparing the
>> >> >>>>>>>>>>>>>>packet number of  ingress NVE and the sum of all egress
>> >> >>>>>>>>>>>>>>NVEs, packet loss can be deduced.
>> >> >>>>>>>>>>>>>> This method can be applicable for both unicast and
>> >> >>>>>>>>>>>>>>multicast  traffic. Local policy on ingress NVE is
>> >> >>>>>>>>>>>>>>configured by  operators or automatically acquired from
>> >> >>>>>>>>>>>>>>centralized orchestration.
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Thanks
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> weiguo
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> _______________________________________________
>> >> >>>>>>>>>>>>>> nvo3 mailing list
>> >> >>>>>>>>>>>>>> [e-mail]
>> >> >>>>>>>>>>>>>> https://www.ietf.org/mailman/listinfo/nvo3
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>
>> _______________________________________________
>> >> >>>>>>>>>>>>>> nvo3 mailing list
>> >> >>>>>>>>>>>>>> [e-mail]
>> >> >>>>>>>>>>>>>> https://www.ietf.org/mailman/listinfo/nvo3
>> >> >>>>>>>>>>>> _______________________________________________
>> >> >>>>>>>>>>>> nvo3 mailing list
>> >> >>>>>>>>>>>> [e-mail]
>> >> >>>>>>>>>>>> https://www.ietf.org/mailman/listinfo/nvo3
>> >> >>>>>>>>> _______________________________________________
>> >> >>>>>>>>> nvo3 mailing list
>> >> >>>>>>>>> [e-mail]
>> >> >>>>>>>>> https://www.ietf.org/mailman/listinfo/nvo3
>> >> >>>>>>>>
>> >> >>>>>>>> _______________________________________________
>> >> >>>>>>>> nvo3 mailing list
>> >> >>>>>>>> [e-mail]
>> >> >>>>>>>> https://www.ietf.org/mailman/listinfo/nvo3
>> >> >>>>>>> _______________________________________________
>> >> >>>>>>> nvo3 mailing list
>> >> >>>>>>> [e-mail]
>> >> >>>>>>> https://www.ietf.org/mailman/listinfo/nvo3
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>
>> >> >>>>> _______________________________________________
>> >> >>>>> nvo3 mailing list
>> >> >>>>> [e-mail]
>> >> >>>>> https://www.ietf.org/mailman/listinfo/nvo3
>> >> >>>
>> >> >>>_______________________________________________
>> >> >>>nvo3 mailing list
>> >> >>>[e-mail]
>> >> >>>https://www.ietf.org/mailman/listinfo/nvo3
>> >> >>
>> >> >>_______________________________________________
>> >> >>nvo3 mailing list
>> >> >>[e-mail]
>> >> >>https://www.ietf.org/mailman/listinfo/nvo3
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > nvo3 mailing list
>> >> > [e-mail]
>> >> > https://www.ietf.org/mailman/listinfo/nvo3
>> >> >
>> >> _______________________________________________
>> >> nvo3 mailing list
>> >> [e-mail]
>> >> https://www.ietf.org/mailman/listinfo/nvo3
>> > _______________________________________________
>> > nvo3 mailing list
>> > [e-mail]
>> > https://www.ietf.org/mailman/listinfo/nvo3