Re: [Bier] I-D Action: draft-kumarzheng-bier-ping-00.txt

["Nobo Akiya" <[e-mail]>]

Hi Eric, Tony, Nagendra, et al,

Great comments and discussions!

I have further comments/questions. Please see in-line with [NOBO].

> -----Original Message-----
> From: Nagendra Kumar Nainar (naikumar) [mailto:[e-mail]]
> Sent: March-12-15 7:21 PM
> To: Eric C Rosen; Antoni Przygienda; [e-mail]
> Cc: Gregory Mirsky; Carlos Pignataro (cpignata); Mach Chen; Nobo Akiya;
> [e-mail]
> Subject: Re: [Bier] I-D Action: draft-kumarzheng-bier-ping-00.txt
> 
> Hi Eric,
> 
> Thanks for the excellent comments. Please see below,
> 
> >I have a few questions and comments.
> >
> >First, thanks for not calling it "bier-pong" ;-)
> 
> <Nagendra> LOL :). When I tried to locate the draft using ³bier + ping² in
google,
> it pops a messages that an unusual search and asked me to confirm that I
am
> human and not machine :)

[NOBO] "bier-pong" is a great name! It's not too late to change :)

> 
> >
> >Any TLV that has a BitString and an SI also has to have a sub-domain
> >id, or else the set of BFERs identified in the BitString cannot be
> >determined (nor can the BFIR).
> 
> <Nagendra> Yes, I agree. We will include sub-domain id in the TLVs.
> 
> >
> >I don't understand why it makes sense to use the BIER data plane to
> >respond to a ping or traceroute.  I'd think you'd want to avoid BIER in
> >the reverse direction, and use ICMP to send back the responses.
> 
> <Nagendra> We considered both. You are right that we don¹t validate
anything
> in the reverse direction and can be IP based reply.

[NOBO] -00 proposes a field "Reply Mode" in the BIER OAM header which allows
an initiator of the operation to choose the reverse "channel" that responder
BFR uses.

                   Value      Meaning
                   --------  ---------------
                     1        Do not Reply
                     2        Reply via IPv4/IPv6 UDP packet.
                     3        Reply via BIER packet

This provides improved diagnostic capability (e.g., we can still diagnose
the BIER layer even if there's a temporary/persistent problem at the IP
layer), and I think this is a good thing. By "I'd think you'd want to avoid
BIER in the reverse direction ...", if there's a technical reason why we
should remove the "Reply via BIER packet" option, I'd like to hear the
reason.

> 
> >This
> >would mean that the request needs to have the IP address to which to
> >respond.  (If you do use BIER for the responses, you'd certainly need
> >to know the sub-domain in which to interpret the BFIR, and you'd need
> >to know the sub-domain to choose the right BIER-MPLS label.)
> 
> <Nagendra> By including the sub-domain in the TLV, we can use the BFIR id
to
> fetch the BFR-Prefi form IGP database. But I think, it will be more clean
to have
> a TLV carrying the ³Reply-to² address which will more control to instruct
what
> address to reply to.
> 
> >
> >Overall, I agree with Tony that the draft would benefit from a lot more
> >attention to procedures -- the document really doesn't make clear just
> >how you put together the set of TLVs to actually do a traceroute (or
> >even a ping).  Until it's clear just what information needs to be
> >carried forwards and back, there's no point in discussing TLV formats.
> 
> <Nagendra> Sure. We will detail the procedure in next revision.
> 
> >
> >For the traceroute case, the draft also needs to address the issue of
> >how the initiator should collate the replies to figure out what path
> >actually gets followed by a given data packet.
> 
> <Nagendra> The current draft proposes TLVs to carry in Reply that includes
> - incoming interface, incoming BitString (in the header) and also
downstream
> neighbors to which the reply was sent, along with egress BitString that
will be
> sent to each downstream neighbor. IMHO, Initiator will be able to use
these
> information (along with sender handler) to portrait end to end path. How
it
> portrait the path would be a local matter, I think. We will include a
point in the
> procedure and mention it as local matter.
> 
> Do you think something is missing from the Initiator point of view that
needs to
> be clarified?.

[NOBO] The BIER OAM header has "Sequence Number" field which is set by the
initiator and incremented with every packet sent, and the responder BFRs
return this value the in reply. The initiator can use this "Sequence Number"
to correlate the ping to pong(s). How an initiator will do this is an
implementation matter, but I do think it's a good idea to capture the
Traceroute and TreeTrace flows in the document.

> 
> >
> >If a user says, "I'm getting spotty service when I try to receive a
> >given data stream", how would one use these mechanisms to troubleshoot?
> 
> <Nagendra> Ok. I think the procedure section (after updating) will be more
> generic. Do you think, it is reasonable to add an Appendix section to give
a
> couple of such examples from usage point of view?.

[NOBO] IMO, the critical aspect is that the implementation will need to take
the user specified <data stream> and generate the BIER header including the
Entropy field. The BIER OAM (i.e., Ping and Traceroute) is then ran on the
generated BIER header to troubleshot, which should traverse exactly the same
ECMP path as the <data stream> does.

> 
> >
> >If you wanted to troubleshoot a problems with a particular user data
> >stream, I think you'd need to know the entropy value of the data
> >packets in the troublesome data stream.  You'd want to specify the
> >entropy and find out what path is followed by packets to a given BFER
> >(or set of
> >BFERs) with that entropy value.  To enable this, the BitString TLVs
> >would need to have an entropy value.  The draft doesn't seem to have
> >any provision for that.  (And unless deterministic ECMP is used, I'm
> >not sure this is a tractable problem at all.)
> 
> <Nagendra> I think, we can achieve this in 2 ways. One option is to have
the
> Initiator (assuming that is also acting as the ingress for the actual data
stream)
> to use the same Entropy value (that will be used in the data
> traffic) in probe packet. By this, we don¹t need to query each transit
BFRs. The
> other option is to query each BFR along the path about entropy range for
each
> ECMP path. This query does not need to carry any data stream details as
the
> entropy query is not specific to stream instead the path that will be
selected
> within a range. This is analogous to ECMP tree trace in MPLS LSP Ping.

[NOBO] IMO, I think the former is most usable. It means that the BIER OAM
CLI/API will allow users to specify the <data stream>, but I don't think the
BitString TLVs need to have any entropy values since the BIER header already
has the Entropy field (draft-wijnands-mpls-bier-encapsulation).

> 
> >
> >For traceroute, I guess the request sent by the initiator would be
> >carried unchanged until there's a TTL expiry.  I think the request
> >needs to carry the original BIER header (augmented with SI and
> >sub-domain); this would be sent back in the response, along with the
> >corresponding information that is in the BIER header when the request
> >is received by the responder.  This would tell you which BFERs from the
> >original BitString are in the sub-tree of the responder.  (The document
> >does have a TLV for this.)
> 
> <Nagendra> Yes.
> >
> >The draft seems to be trying to have the mechanisms discover all
> >possible ECMP paths that a packet might take, but I don't really see
> >how that is going to work in a scalable manner.
> 
> <Nagendra> The primary intention is to have a way to perform ECMP query
per
> BFER. To troubleshoot a failure to a specific BFER or set of BFERs should
be
> possible by querying (and using) ECMP paths for one of the affected BFER
and
> use the entropy.
> 
> 
> >Also, I don't understand
> >the proposed use of the "Multipath Entropy TLV".  What value gets put
> >in it, and under what circumstances?
> 
> <Nagendra> This is similar to section 3.3.1 of RFC4379. To avoid redundant
> information, we tried referring to RFC4379 (noticed that section info is
not
> given). Will it be helpful to have an example in Appendix?.
> 
> >
> >I don't really understand the Upstream Interface and Downstream Mapping
> >TLVs.
> 
> <Nagendra> Upstream Interface TLV is used by the responder to define the
> incoming interface of the probe packet. This is useful to detect any
deviation in
> HW/distributed forwarding. Per BFR-A control-plane, Bit-id 10 is reachable
via
> interface-A  while the data plane have a stale entry pointing towards
Interface-B.
> In this case, the reply from BFR-A will have downstream address which does
not
> match with upstream interface TLV of next response.

[NOBO] It is really the Downstream Mapping TLVs that provides the Traceroute
operation to perform the validation at every transit BFR. A BFR at TTL=n
will say outgoing adjacency is X in the Downstream Mapping TLV included in
the Reply. Initiator takes the Downstream Mapping TLV received from TTL=n in
the Reply and places the same Downstream Mapping TLV in the Request for
TTL=n+1. The BFR at TTL=n+1 will take the received Downstream Mapping TLV
and validates that the packet was indeed received on the "place" which is
described by the Downstream Mapping TLV.

Upstream Interface TLV is for every BFR to return the "place" which the BIER
packet was received from, so that initiator, thus user, has more information
of actual paths traversed in case of "problems".

> 
> 
> > If these are supposed to identify BIER adjacencies, then they need to
> >take account of the fact that a pair of adjacent BFRs might be
> >separated by an IP or MPLS tunnel, rather than being directly connected.
> >  Or are these TLVs supposed to identify IGP adjacencies rather than
> >BIER adjacencies?  What exactly is the intended function of these TLVs,
> >and will they fulfill that function when the adjacenct BFRs are not
> >directly connected?
> 
> <Nagendra> Good point about the IP/MPLS tunnel :).The intention is to
carry the
> BIER adjacency details. We will check this point and include in next
revision.

[NOBO] Does a pair of BFRs separated by an IP or MPLS tunnel still have a
direct BIER adjacencies at the BIER layer? If yes (and I think this is the
case, but do correct me if I'm wrong), then I think we are good as long as
we place clarifications in the OAM document.

> 
> >
> >With regard to traceroute, have you considered the impact of the
> >"uniform model" for TTL handling (see RFC 3443)?  It may not always be
> >possible to ensure that the TTL in the BIER-MPLS label is decremented
> >only by the number of BFR hops.  In such a case, once a request has
> >reached a particular BFR, the initiator might have to increment TTL
> >several times to get a request to reach the next BFR hops.  This is
> >something that needs to be discussed in a specification of the
> >traceroute procedures.  It's also worth thinking about what would
> >happen if a BIER OAM packet experiences TTL expiry while in the middle
> >of a unicast tunnel.
> 
> <Nagendra> Yes, Right. This needs to be clarified in the document. We will
> include this in next revision.

[NOBO] Very good point. For the "uniform model", it might be safer so say
that the Tracerout operation cannot be supported, i.e., trying to tweak the
TTL at BFRs is error prone and can likely create more problems (e.g., tunnel
reopts, FRR, etc). Unless of course the BIER WG is ok placing a TTL/HopCount
in the BIER Header :)

> 
> >
> >The request has a "sequence number" that is supposed to be used to
> >detect missing responses.  I don't see how that will work, since a
> >single request in general elicits multiple responses.  You wouldn't be
> >able to use the sequence number to detect missing responses unless all
> >responses to the request with that sequence number were missing.
> >
> >It might make sense though for the sequence number to be the same as
> >the original TTL placed in the BIER-MPLS label by the initiator of the
> >request.
> 
> <Nagendra> Thanks for the alternate suggestion. But I think it will suffer
the
> same problem. I will discuss this with other authors and update
accordingly.

[NOBO] Hmm... 

For Ping, number of responses == number of 1s in the BitString.

For Traceroute, TTL=n tells you how many downstreams there are. Thus number
of responses at TTL=n+1 == number of downstreams specified by TTL=n.

One thought triggered in my head as result is above comment, for TreeTrace,
is that the "downstreams" described by the BFR Reply need to differentiate
between replicated downstreams vs. ECMPed downstreams.

Thanks!

-Nobo

> 
> >
> >The return codes are not very well-defined, and it's not clear just
> >what the principles are that led to choosing that particular set of
> >return codes.  It would be good to see some discussion of what
> >information it is useful to return in various troubleshooting scenarios.
> 
> <Nagendra> Sure. We will update with more details on the Return code.
> 
> Once again, thanks for the wonderful comments.
> 
> Regards,
> Nagendra