Re: [ippm] v6 option types for IOAM data fields

[Tom Herbert <[e-mail]>]

On Thu, Nov 8, 2018, 12:37 AM Frank Brockners (fbrockne) <[e-mail]>
wrote:

> Hi Mark, Mike,
>
>
>
> thanks for summarizing the concerns around leaked packets – and outlining
> an approach to mitigate those. There are a couple of other challenges we
> face. Unfortunately we did not get to a discussion in the 6man WG meeting
> this time due to the limited time we had. The meeting slides
> https://datatracker.ietf.org/meeting/103/materials/slides-
> 103-6man-in-situ-oam-ipv6-options-00 provide a summary of concerns.
>
>
>
> In a nutshell:
>
> 1.       Potential HbH ext header and IOAM option insertion and removal
> by transit nodes (i.e. “en route”) in a restricted administrative domain.
>
> a)       How do you deal with PMTU? – Packet size changes might exceed
> PMTU.
>
> b)      Misleading ICMP errors confusing the source
>
> c)       Possible leaks that affect the forwarding behavior and state of
> network elements outside the domain.
>

There's already been significant discussion in 6man list about option
insertion/removal. I believe the conclusion is that RFC8200 prohibits it
exactly for the reasons listed above and maybe a few others. I don't think
that restricting the use to a limited domain is sufficient rationale to
create an exception. As mentioned previously, encapsulation can be used to
achieve the proper effect.

> 2.       Incremental Trace IOAM HbH Option – which is to support a
> hardware-friendly implementation: Changes Option Data Len en-route.
>
> a)       Dealing with PMTU– Packet size changes can exceed PMTU; see above
>

This has basically the same problems as option insertion. RFC8200 only
allow Option Data to be changeable en route, and that is only if the third
bit of the option type is set.

Here are two other issues for these:

- No accountability as to who changed the packet in flight
- They break authentication header

> 3.       Clarify applicability statement and scope
>
>
>
> While 3. is something that is in the works, there are no easy answers to
> 1. and 2. Considerations:
>
>
>
> On 1. Supporting HbH ext header and option insertion/removal in transit. A
> couple of solution approaches…
>
> 1.       Fix PMTU and offset for packet size change in PMTU discovery -
> https://tools.ietf.org/html/draft-troan-6man-pmtu-solution-space-00
> Hope that we can make progress towards a solution tomorrow…
>
> 2.       IP-in-IP with IOAM extension header in the **inner** packet.
> New IPv6 packet is created with encapsulating node as source and the
> original destination as the destination
> (this is slightly different than what you, Mark, suggest – because we’d
> like to keep the forwarding path the same. Tunneling with ULA would not get
> us there; the slides above have a diagram that depicts the potential encap):
>
> a)       Payload of this packet is the original IPv6 packet along with an
> extension header inserted inside.
>
> b)      The original packet is restored by removing the outer IPv6 header
> and the inner extension header by a node at the domain boundary.
>
> c)       Caveats:
>
>                                                                i.      Modified
> packet may still leak – but will only confuse the destination node. The
> dest node will likely send an ICMP back to the encap node, which gets the
> encap node an understanding that something is going wrong.
>
>                                                              ii.      ECMP
> computation needs to be reworked.
>
>                                                            iii.      Complex/costly
> implementation in HW & SW – IOAM-capable nodes need to hunt for the IOAM
> data fields in the inner packet.
>
> 3.       No support of IOAM in transit network. Support only source
> initiated IOAM tracing, proof of transit..
> Caveat: Limits usage of IOAM significantly.
>
>
>
> On 2. Incremental Trace IOAM HbH Option
>
> 1.       Use PMTU to determine max possible Incremental Trace IOAM Option
> length
>
> 2.       Do not support Incremental Trace IOAM Option in IPv6.
>
>
>
> While there isn’t an easy answer, IMHO we should still try to find a
> workable and implementable solution. I sometimes compare the situation with
> the road network: What we have today is a network of gravel roads (Internet
> with often poor ext header processing) and cars are built to cope with
> gravel roads. Now faster and comfy cars become available, though they only
> work on tared/concrete roads (specific administrative domains). If you try
> to drive these new cars on gravel roads they break and the wrecks jam the
> roads…, still we probably want to allow folks to buy and use the fast &
> comfy cars, because they will also push for better roads  as well (do
> proper ext header processing).
>
>
>
> Thoughts?
>
>
>
> Thanks, Frank
>
>
>
>
>
> *From:* Mark Smith <[e-mail]>
> *Sent:* Dienstag, 30. Oktober 2018 05:26
> *To:* Frank Brockners (fbrockne) <[e-mail]>
> *Cc:* C. M. Heard <[e-mail]>; 6man WG <[e-mail]>;
> [e-mail]
> *Subject:* Re: v6 option types for IOAM data fields
>
>
>
>
> Hi Frank,
>
>
>
> On Tue., 30 Oct. 2018, 05:36 Frank Brockners (fbrockne), <
> [e-mail]> wrote:
>
> Thanks Mike. On the scope of an IOAM deployment:
> draft-ietf-ippm-ioam-data-04 clarifies in section 3 that IOAM is a domain
> focused feature, i.e. not expected to be deployed on the open Internet.
>
>
>
> That still violates RFC 8200, there are no exceptions for "closed" domains.
>
>
>
> From section 3:
>
>
>
> "Designers of
>
>    carrier protocols for IOAM must specify mechanisms to ensure that
>
>    IOAM data stays within an IOAM domain.  In addition, the operator of
>
>    such a domain is expected to put provisions in place to ensure that
>
>    IOAM data does not leak beyond the edge of an IOAM domain, e.g. using
>
>    for example packet filtering methods."
>
>
>
> Neither the designers or the operators can ensure this IOAM information
> won't leak.
>
>
>
> Possible reasons it can leak:
>
>
>
> - operator configuration error - e.g. forgetting to configure the domain
> boundary (e.g. router with multiple domain exit interfaces, forgetting to
> add that option on one of them), or misconfiguring it.
>
>
>
> - partial device failure - the device still forwards packets, but ceases
> to remove this information due to a hardware fault that has appeared
>
>
>
> - vendor implementation bugs that fails to remove the information.
>
>
>
> Any or all of these can occur, and as it is on egress, the consequences
> may not be visible to the domain network operator, because network
> operators rarely inspect packets after they've left their network. Once a
> packet is sent onto somebody else, it is assumed to have been sent without
> faults.
>
>
>
> There are many instances of leaks at the boundary of what are supposed to
> be closed domains, such as packets containing RFC1918 private addresses (in
> packet headers themselves, or in DNS packets - such a big problem that www.a
> s112.net <http://as112.net> was created), and route leaks e.g.
> https://bgpmon.net/how-the-internet-in-australia-went-down-under/
>
>
>
> As operators usually have only one device at the edge of the domain, that
> would be performing the domain boundary function, that device becomes a
> single point of failure. Enforcement is therefore quite fragile. A domain
> boundary enforced by two domain edge devices in-line would increase
> robustness, however I'd think that would be rarely deployed, because we
> haven't seen that commonly with IPv4 NAT.
>
>
>
> The only way for an operator to ensure these leaks don't and never occur
> is for the domain to literally not be physically connected to the Internet
> i.e. the domain is air gapped from the Internet. If there is a link between
> the domain and the Internet that can carry IP packets, then there is always
> a possibility that information can leak from the domain.
>
>
>
> So accepting that leaks can always occur, a far more robust option is to
> leave the original packet alone entirely, and encapsulate it in a domain
> local IPv6 header (i.e RFC2473, section 3.1) with domain local limited
> addresses (i.e. ULA addresses) and the additional EH.
>
>
>
> That still doesn't ensure that packet won't leak outside the domain,
> because those packets will follow a default route, however as the packet's
> addressing is invalid outside of the domain, that packet will be dropped
> once it reaches either an invalid Internet address filter, or a default
> free Internet router. The packet with the failed-to-be-removed outer ULA
> IPv6 header will never reach the destination address of the inner packet,
> because the DA of the inner packet is in the ULA IPv6 packet's payload.
>
>
>
> Failure of the mechanism would be much more obvious, localised to the
> domain implementing the mechanism (rather than being externalised to
> somebody else's domain/network), and absolute, because the failure mode is
> packets being entirely discarded.
>
>
>
> Regards,
>
> Mark.
>
>
>
>
>
>
> Frank
>
> -----Original Message-----
> From: C. M. Heard <[e-mail]>
> Sent: Montag, 29. Oktober 2018 16:03
> To: 6man <[e-mail]>; IPPM <[e-mail]>
> Cc: Frank Brockners (fbrockne) <[e-mail]>
> Subject: Re: v6 option types for IOAM data fields
>
> On Thu, 25 Oct 2018 15:06:57 +0000 Frank Brockners (fbrockne) wrote:
> > Quick heads up: In the 6MAN meeting in BKK, we’ll review
> > draft-ioametal-ippm-6man-ioam-ipv6-options-01 – which requests 2
> > option types from the DO/HbyH options sub-registry.
> >
> > While the bulk of the IOAM work is progressed in the IPPM WG, we’d
> > greatly appreciate your feedback on
> > draft-ioametal-ippm-6man-ioam-ipv6-options-01,
> > which defines how IOAM data fields are carried using v6 extension
> headers.
> > Cc’ing the IPPM WG as well, to keep everyone on the same page.
>
> I have two brief comments on this work.
>
> First, I see that the Incremental Tracing Option changes length in transit.
> It is not appropriate for it to be carried in an IPv6 option intended for
> use on the open Internet, for exactly the same reason that insertion of
> extension headers by intermediate nodes is not allowed on the open Internet.
>
> Second, I see that two IPv6 option code points are requested, one with the
> "chg" flag set, the other with the "chg" flag clear. While there is no harm
> in this, it is not strictly necessary; the only real purpose of this flag
> is to determine whether the option data is or is not included in the
> Authentication Header Integrity Check Value computation.
>
> Mike Heard
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [e-mail]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------
>
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [e-mail]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------
>