Re: [ippm] draft-ioametal-ippm-6man-ioam-ipv6-deployment-00 feedback (Re: v6 option types for IOAM data fields)

[Tom Herbert <[e-mail]>]

On Fri, Apr 5, 2019, 2:07 PM Jai Kumar <[e-mail]> wrote:

> Tom,
>
> Its all explained in the requirements section of the draft. IFA header is
> essentially treated as an EH both in IPv4 and IPv6 (very similar to AH/ESP
> EH).
>
> Just like in AH/ESP the IP protocol is copied in the AH header and IP
> protocol field is replaced in the IP header with the AH proto.
>
> There are several advantages to this approach.
> - devise know how to parse AH header and can follow very similar approach
> - Protocol agnostics, silicon need not support N encaps (just like
> AH/ESP). Single implementation works for all protocol types.
> - No need to creation options and teach devices that options is NOT an
> exception path processing
> - Really worry about ordering in IPv6 options
> - Note that all the OAM data need to be deleted as well so position need
> to be optimal for removal. We can always do tail stamping of the data but
> this puts considerable constrains on the cell based architecture
> - L4 accessibility I have already mentioned
>
> I am happy to discuss with you in person and explain more but bottom line
> is that this proposal is what came out with MSDC discussions in light of
> shortcomings from other IETF proposals.
>

As they say, you are free to run whatever you want in your proprietary
network, but if you expect to create an interoperable, deployable protocoI
and get a protocol number assignment, you'll need to work in ietf. That is
going to require justifying the protocol even in light of shortcomings in
existing protocoIs.

Tom


> Regards,
> -Jai
>
>
> On 4/5/19, 12:28 PM, "Tom Herbert" <[e-mail]> wrote:
>
>     On Fri, Apr 5, 2019 at 12:03 PM Jai Kumar <[e-mail]>
> wrote:
>     >
>     > " Great, but this IETF"
>     > Does it mean that drafts have no relevance to deployment and actual
> adoption.
>     >
>     > For all your queries please take a look at IFA draft.
>     > https://datatracker.ietf.org/doc/draft-kumar-ippm-ifa/
>     >
>     I've looked at that draft. It's is creating a complete new IP
>     protocolas well as a new packet format for TCP and UDP that somehow
>     inserts meta data between the TCP header and payload. I don't
>     understand why this could possibly be better than just using flow
>     label for ECMP and teaching devices that need to extract transport
>     information how to walk over EH, or just using some UDP encpasulation
>     like in draft-herbert-ipv4-udpencap-eh-01.
>
>     Tom
>
>     > It handles IPv6, IPv4 TCP/UDP with altering the packet path (due to
> inserting options and/or encaping them in GRE header) as well as IPSec
> AH/ESP/WESP both tunnel and transport and guess what it is successfully
> deployed.
>     >
>     > -Jai
>     >
>     > On 4/5/19, 11:54 AM, "Tom Herbert" <[e-mail]> wrote:
>     >
>     >     On Fri, Apr 5, 2019 at 11:25 AM Jai Kumar <
> [e-mail]> wrote:
>     >     >
>     >     > Tom,
>     >     >
>     >     > I am speaking based on the actual deployment not based on if
> all the traffic is/will move to IPSec tunnel.
>     >     >
>     >     > I have 6 large customer deployments and one someone previously
> called as "Yahoo" who insist on visibility in L4 even for IOAM packets.
>     >     >
>     >     And I've worked in datcenters much larger, and my first rule is
> that I
>     >     don't want random router vendors dictating to me what protocols
> I'm
>     >     allowed to use in my datacenter.
>     >
>     >     > I can quote you on that and lose my business :-)
>     >     >
>     >     > ECMP argument is weak as well as it works for IPv6 flow label.
> What about IPv4 TCP/UDP traffic which typically is 80% of the traffic in
> MSDC.
>     >     >
>     >
>     >     draft-herbert-ipv4-udpencap-eh-01
>     >
>     >     > If someone tells me that proposal will work only for my 20% of
> traffic then again as I said before, they will lose my business :-)
>     >     >
>     >     Great, but this IETF. Where are the protocol requirements? If
> you are
>     >     saying that there are requirements for "visibility in L4" what
> are
>     >     they? If you are saying that ACLs are now required for packet
>     >     forwarding, where is that described? Is it a MUST that teh only
>     >     transport protocols we're allowed to use are  UDP or TCP? Are we
>     >     allowed to use extension headers at all? I suppose fragmentation
> is
>     >     right out... Are we violating the requirements if we encrypt the
>     >     transport headers? If you say that we shouldn't send long headers
>     >     because we'll overflow parsing buffers, then what is the limit?
> Please
>     >     be SPECIFC in setting requirements; we are trying to build
> protocols
>     >     that work and are interoperable and we need to get out of this
> morass
>     >     of reverse engineering to discover the least common denominator.
>     >
>     >     Tom
>     >
>     >     > Regards,
>     >     > -Jai
>     >     >
>     >     > On 4/5/19, 11:15 AM, "Tom Herbert" <[e-mail]>
> wrote:
>     >     >
>     >     >     On Fri, Apr 5, 2019 at 10:55 AM Jai Kumar <
> [e-mail]> wrote:
>     >     >     >
>     >     >     > Even if we use flow label for ECMP, lack of
> accessibility of L4 information (for various services, most simple is ACL)
> makes the proposal pretty much unusable.
>     >     >     >
>     >     >     That is convoluting router and firewall functionality and
>     >     >     requirements. This a discussion about packet forwarding
> which does
>     >     >     _not_ require ACLs. In a limited domain, for which IOAM is
> intended,
>     >     >     it is unlikely that they'll ACLs would be used in internal
> nodes, but
>     >     >     we do need good ECMP routing at all intermediate nodes
> need to be
>     >     >     consistent rather or not that the IOAM option is present.
> As side
>     >     >     note, the ability to extract L4 information out of packets
> is
>     >     >     dwindling anyway thanks to encryption and other techniques
> trying to
>     >     >     undo protocol ossification. For instance, good luck
> getting L4
>     >     >     information out of an IPsec tunnel :-).
>     >     >
>     >     >     Tom
>     >     >
>     >     >     > NIC cards are not used pervasively for ACLs and other
> services, I believe iptables in kernel are good enough for host but that’s
> not acceptable in networking elements.
>     >     >     >
>     >     >     > -Jai
>     >     >     >
>     >     >     > On 4/5/19, 10:31 AM, "ippm on behalf of Tom Herbert" <
> [e-mail] on behalf of [e-mail]> wrote:
>     >     >     >
>     >     >     >     On Fri, Apr 5, 2019 at 9:16 AM Pengshuping (Peng
> Shuping)
>     >     >     >     <[e-mail]> wrote:
>     >     >     >     >
>     >     >     >     > Hi Barak,
>     >     >     >     >
>     >     >     >     > We are certainly not designing for the lowest
> capable solutions but exploring optimal solutions which could help to keep
> the forwarding performance while inserting the iOAM data with variable
> length.
>     >     >     >
>     >     >     >     Shuping,
>     >     >     >
>     >     >     >     That's exactly the argument for using the flow label
> in ECMP.
>     >     >     >     Efficeint packet forwarding can be done solely by
> inspected the forty
>     >     >     >     byte fixed length header of the packet regardless of
> the contents of
>     >     >     >     the rest of the packet. That is an optimal solution.
> Anything else for
>     >     >     >     ECMP like doing DPI or hacking up IOAM to try do
> pull transport layers
>     >     >     >     into the parsing buffer (whatever size that may be)
> are nothing more
>     >     >     >     than hacks and unnecessarily perpetuate techniques
> used with IPv4 that
>     >     >     >     either don't work or are unnecessary in IPv6.
>     >     >     >
>     >     >     >     Tom
>     >     >     >
>     >     >     >
>     >     >     >     >
>     >     >     >     > In terms of the forwarding efficiency, we would
> all agree that a short and fixed header would be better than a variable
> header.
>     >     >     >     >
>     >     >     >     > Best regards
>     >     >     >     > Shuping
>     >     >     >     >
>     >     >     >     >
>     >     >     >     > 发件人： Barak Gafni<[e-mail]>
>     >     >     >     > 收件人： Pengshuping (Peng Shuping)<
> [e-mail]>;Frank Brockners (fbrockne)<[e-mail]>;Mikael
> Abrahamsson<[e-mail]>
>     >     >     >     > 抄送： C. M. Heard<[e-mail]>;6man WG<
> [e-mail]>;Mark Smith<[e-mail]>;ippm<[e-mail]>
>     >     >     >     > 主题： RE:
> draft-ioametal-ippm-6man-ioam-ipv6-deployment-00 feedback (Re: v6 option
> types for IOAM data fields)
>     >     >     >     > 时间： 2019-04-05 03:18:26
>     >     >     >     >
>     >     >     >     > Hi Shuping,
>     >     >     >     > I think that the question is whether we design for
> the lowest capable solutions. I assume someone can find solutions with even
> lower amount of parsing.. The low end solutions for these capabilities may
> need to either adopt with new designs or use solutions such as what is
> suggested in this thread by Tom Herbert.
>     >     >     >     > Personally, I don't think we should go this way,
> but to design appropriate solutions, with appropriate layers in the
> headers. For example, prevent dependencies between "remote" headers.
>     >     >     >     >
>     >     >     >     > Thanks,
>     >     >     >     > Barak
>     >     >     >     >
>     >     >     >     > -----Original Message-----
>     >     >     >     > From: ippm <[e-mail]> On Behalf Of
> Pengshuping (Peng Shuping)
>     >     >     >     > Sent: Wednesday, April 3, 2019 6:47 AM
>     >     >     >     > To: Frank Brockners (fbrockne) <[e-mail]>;
> Mikael Abrahamsson <[e-mail]>
>     >     >     >     > Cc: C. M. Heard <[e-mail]>; 6man WG <
> [e-mail]>; Mark Smith <[e-mail]>; [e-mail]
>     >     >     >     > Subject: [ippm] 答复:
> draft-ioametal-ippm-6man-ioam-ipv6-deployment-00 feedback (Re: v6 option
> types for IOAM data fields)
>     >     >     >     >
>     >     >     >     > Hi,
>     >     >     >     >
>     >     >     >     > The parsing depth of the early chips is 96B/128B.
> For the new chips the parsing depth has been increased but still limited.
> So Mikael's concern makes sense especially in the tracing mode that the
> IOAM data fields are going to increase significantly along the path, which
> will even push the Routing Header out of the parsing depth of the chip. So
> it would make sense to split the IOAM data part from the IOAM
> header/instruction part, and place the IOAM data after the RH or even after
> the L4 header in order to be able to read the L4 information to enable
> ECMP, as stated in the draft
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Fdraft-li-6man-ipv6-sfc-ifit-00&amp;data=02%7C01%7Cgbarak%40mellanox.com%7C612ba6dc117440e68c4608d6b83ae174%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636898960379485833&amp;sdata=Dob4zN%2F1jSGxLTlqGma29JJNvqJDBt4VzqHsN4OpxM4%3D&amp;reserved=0
> .
>     >     >     >     >
>     >     >     >     > BR,
>     >     >     >     > Shuping
>     >     >     >     >
>     >     >     >     >
>     >     >     >     > -----邮件原件-----
>     >     >     >     > 发件人: ippm [mailto:[e-mail]] 代表 Frank
> Brockners (fbrockne)
>     >     >     >     > 发送时间: 2019年4月2日 17:40
>     >     >     >     > 收件人: Mikael Abrahamsson <[e-mail]>
>     >     >     >     > 抄送: C. M. Heard <[e-mail]>; 6man WG <
> [e-mail]>; Mark Smith <[e-mail]>; [e-mail]
>     >     >     >     > 主题: Re: [ippm]
> draft-ioametal-ippm-6man-ioam-ipv6-deployment-00 feedback (Re: v6 option
> types for IOAM data fields)
>     >     >     >     >
>     >     >     >     >
>     >     >     >     >
>     >     >     >     > -----Original Message-----
>     >     >     >     > From: Mikael Abrahamsson <[e-mail]>
>     >     >     >     > Sent: Dienstag, 2. April 2019 08:06
>     >     >     >     > To: Frank Brockners (fbrockne) <[e-mail]
> >
>     >     >     >     > Cc: Mark Smith <[e-mail]>; C. M.
> Heard <[e-mail]>; 6man WG <[e-mail]>; [e-mail]
>     >     >     >     > Subject: RE:
> draft-ioametal-ippm-6man-ioam-ipv6-deployment-00 feedback (Re: v6 option
> types for IOAM data fields)
>     >     >     >     >
>     >     >     >     > On Mon, 1 Apr 2019, Frank Brockners (fbrockne)
> wrote:
>     >     >     >     >
>     >     >     >     > > ...FB: There is obviously no easy answer. Couple
> of thoughts:
>     >     >     >     > > * IOAM is considered a "domain specific" feature
> (see
>     >     >     >     > > draft-ietf-ippm-ioam-data-05, section 3).
> Routers in the IOAM domain
>     >     >     >     > > should be IOAM capable.  And IMHO,  we should
> add a statement to
>     >     >     >     > > draft-ioametal-ippm-6man-ioam-ipv6-deployment
> that an implementation
>     >     >     >     > > of IOAM MUST ensure "C1".
>     >     >     >     > >
>     >     >     >     > > * That said, there can be situations, where C1
> cannot be achieved, e.g..
>     >     >     >     > > consider a network which does ECMP scheduling
> based on packet length.
>     >     >     >     > >
>     >     >     >     > > * What one could consider - and which is one
> suggested deployment
>     >     >     >     > > model
>     >     >     >     > > - is that by default IOAM data fields are added
> to _all_ customer
>     >     >     >     > > packets. The decapsulating node would decide
> whether the IOAM
>     >     >     >     > > information contained in a packet would be used
> (and exported) or not.
>     >     >     >     > > That way one would not need to deal with the
> situation that some
>     >     >     >     > > traffic carries IOAM while other does not - and
> might thus be treated
>     >     >     >     > > differently.
>     >     >     >     >
>     >     >     >     > Yes, I think this is the only way. Is there a risk
> that intermediate routers would not be IOAM capable? I think the C1
> requirement is really really hard to fulfil and I'm also afraid that adding
> the IOAM header will actually make ECMP stop working on some platforms
> (because they would not have the capability to look deep enough into the
> packet to find L4 information, so it'll go back to 2 tuple ECMP instead of
> 5 tuple.
>     >     >     >     >
>     >     >     >     > But this question can only be answered by people
> with deep NPU knowledge...
>     >     >     >     >
>     >     >     >     > .....FB2: Given that IOAM is a domain specific
> feature - I expect that folks initially start to use it in domains (like
> e.g. a DC) where all boxes are IOAM capable and can trace the packet
> appropriately - or per Tom's note, can be configured so that one would do
> ECMP based on addresses and flow-label.  There are chips with pretty deep
> parsing capability (up to a few hundred bytes).
>     >     >     >     >
>     >     >     >     > > ...FB: The comparison to MPLS is interesting.
> How often do MPLS
>     >     >     >     > > packets leak and cause harm? For IOAM,
>     >     >     >     > > draft-ioametal-ippm-6man-ioam-ipv6-options-02
> proposes a deployment
>     >     >     >     > > model similar to what we do for MPLS: Unless an
> interface is
>     >     >     >     > > explicitly configured for IOAM, packets with
> IOAM data fields MUST be dropped.
>     >     >     >     > > Hence leakage would only occur, if we have a
> clearly misbehaving
>     >     >     >     > > router which violates this rule. Similar to you,
> I'd also greatly
>     >     >     >     > > appreciate any pointers to research on how
> common MPLS leaked packets are.
>     >     >     >     >
>     >     >     >     > When it comes to "cause harm" I imagine there are
> (at least) two ways to cause harm, one is privacy/secrecy/security loss and
> the other one is actual operational loss.
>     >     >     >     >
>     >     >     >     > I know of bugs where labeled packets went the
> wrong way and caused them to be lost, I've also seen bugs where bugs caused
> traffic to "hop" between VRFs in MPLS VPN (or to "global" VRF). I don't
> have numbers on this though.
>     >     >     >     >
>     >     >     >     > Depending on the deployment scenario, it might
> make sense to make IOAM packets not valid for non-IOAM aware devices
> (basically encap entire packet/payload), but that might be a problem for
> intermediate non-IOAM nodes then. This would affect the ECMP requirement. I
> can see cases where one would operationally turn on IOAM for some customers
> traffic and then see the problem go away because now ECMP changed.
>     >     >     >     >
>     >     >     >     > > ...FB: One idea that Shwetha came up with to
> identify the source AS of
>     >     >     >     > > a leaked packet, would be to add a new new IOAM
> E2E option - as
>     >     >     >     > > proposed in section 5.1.1 bullet 2 of
>     >     >     >     > > draft-ioametal-ippm-6man-ioam-ipv6-deployment-01.
>     >     >     >     >
>     >     >     >     > Yes, that'd solve that problem.
>     >     >     >     >
>     >     >     >     > How much has the silicon people been involved so
> far in the design of the headers? What is the current thinking of amount of
> data going into the IOAM header? Considering things like trace options etc
> it seems to me the header can grow quite large? To satisfy the ECMP
> requirement what about putting the IOAM information as a trailer behind the
> regular payload? Or is there a problem for the hw to manipulate information
> that far into the packet (I also imagine this will considerably lower the
> forwarding performance of IOAM packets on IOAM aware platforms).
>     >     >     >     >
>     >     >     >     > .....FB2: There are quite a few folks with
> hardware background that co-author the IOAM drafts. Parsing capability
> differs between chips, as does the ability to deal with new/flexible
> headers and the ability to modify data in the extension headers. So the
> unfortunate answer to that question is "it depends" (what information do
> you gather, how large is your domain, what are the capabilities of your
> hardware/software forwarder, etc.), but I do expect that for specific
> deployment domains (e.g. DC, Enterprise) - but also for overlay networks /
> VPNs - we'll see an increasing amount of chips that are well suited for
> processing large extension header.
>     >     >     >     >
>     >     >     >     > Cheers, Frank
>     >     >     >     >
>     >     >     >     > --
>     >     >     >     > Mikael Abrahamsson    email: [e-mail]
>     >     >     >     >
>     >     >     >     > _______________________________________________
>     >     >     >     > ippm mailing list
>     >     >     >     > [e-mail]
>     >     >     >     >
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fippm&amp;data=02%7C01%7Cgbarak%40mellanox.com%7C612ba6dc117440e68c4608d6b83ae174%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636898960379485833&amp;sdata=VjZ1PiwifYGCf159je0yef3BJFz6AQNNAtkgmAMDFKM%3D&amp;reserved=0
>     >     >     >     > _______________________________________________
>     >     >     >     > ippm mailing list
>     >     >     >     > [e-mail]
>     >     >     >     >
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fippm&amp;data=02%7C01%7Cgbarak%40mellanox.com%7C612ba6dc117440e68c4608d6b83ae174%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636898960379485833&amp;sdata=VjZ1PiwifYGCf159je0yef3BJFz6AQNNAtkgmAMDFKM%3D&amp;reserved=0
>     >     >     >     >
> --------------------------------------------------------------------
>     >     >     >     > IETF IPv6 working group mailing list
>     >     >     >     > [e-mail]
>     >     >     >     > Administrative Requests:
> https://www.ietf.org/mailman/listinfo/ipv6
>     >     >     >     >
> --------------------------------------------------------------------
>     >     >     >
>     >     >     >     _______________________________________________
>     >     >     >     ippm mailing list
>     >     >     >     [e-mail]
>     >     >     >     https://www.ietf.org/mailman/listinfo/ippm
>     >     >     >
>     >     >     >
>     >     >     >
>     >     >
>     >     >
>     >     >
>     >
>     >
>     >
>
>
>
>