Re: [ippm] Review on draft-ietf-ippm-ioam-data-06

Tom Herbert <tom@quantonium.net> Thu, 22 August 2019 21:11 UTC

Return-Path: <tom@quantonium.net>
X-Original-To: ippm@ietfa.amsl.com
Delivered-To: ippm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 538B112012A for <ippm@ietfa.amsl.com>; Thu, 22 Aug 2019 14:11:44 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.897
X-Spam-Level:
X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=quantonium-net.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EZRUJ3C6mAYp for <ippm@ietfa.amsl.com>; Thu, 22 Aug 2019 14:11:41 -0700 (PDT)
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 20B5A1209D8 for <ippm@ietf.org>; Thu, 22 Aug 2019 14:11:41 -0700 (PDT)
Received: by mail-ed1-x529.google.com with SMTP id f22so9991552edt.4 for <ippm@ietf.org>; Thu, 22 Aug 2019 14:11:41 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quantonium-net.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=8Wu7Zau4jTwTAkxVexjS+623/T80c1N9BkuIVKbX/ig=; b=cTS4MMO43521GY7qpjM2Ciu6PxXjZaJYwK0YW7BMVtPKaggAru9EI2V/S/mWk09k0J rLFotX5U8e1tSR5/syO3xkQsDLJ4ti1r9oRqSUIRzP/6GdTocrrXss2w+gtb/f3AZPnx dzZR09v0lbhk0mQ0UaShITUcwXppl73aFpm2iZDTa8H1nrMYY07Azl/wCv2Ok5XyHxj2 JUtY60aQxc2fiIe+DCdcx7adhF5vK9Zu4iKUB5WUaoz2mK5alI3uCljrmgnjso8C+wD0 YNnqvaarNan1BFdjUXQJ3HPBppL8Ko3+YPGWMtrFTdTZUt3Twa3gy4TAIRGhoqEf32T2 f06w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=8Wu7Zau4jTwTAkxVexjS+623/T80c1N9BkuIVKbX/ig=; b=WKYyrFgajLBbnsVH0DMUDHiY6JIFumwqEF7SrR82RTvsTaPJtT6ZLag9gIfnzzQz4N hpKPGl1+s4lXRo0ve+LP2sXdP53HVHc9DPryZ+7E2UyIHx0wwhf0RZvfGm1GE3yI9aGv NDr+uFtuYBf4cj0ru1f9FIWdsG5Xah7/Wo17B+iSwMuT514ndjn/glW/6Z6laz8M4BD+ wfjRYasLli2a+zK6ToK3dHJPpd0AlV9rVUW9WmE53kysRUDZgF9BP6H0S63I5Dj1ctIH 08YYwmGoEU0mnN1NRFeRa33Eb+VulSmaOeYjUMnIedoLOG3Diy3Ayd4YE2XC+0UN4Sg3 bIAA==
X-Gm-Message-State: APjAAAVEMe8oSNxfTzJ5DAikbnl9Jro4WV58OL4DUIXupNk6js6nbKg0 84sD0vvdD+2AhPvmFER85m0xwe1hdMySKWoitfktVQ==
X-Google-Smtp-Source: APXvYqyw7rsfNIwVHRZY9JrSOkGmcF7Pqa7EWHBtBT31HwdxDn1aal9fdGrKZ0MIcqQEai4xhpKPP0OPkrReBDoDUGI=
X-Received: by 2002:a17:906:cc81:: with SMTP id oq1mr1278599ejb.124.1566508299378; Thu, 22 Aug 2019 14:11:39 -0700 (PDT)
MIME-Version: 1.0
References: <B5A76AB5-AE39-4771-9472-38454CF52152@broadcom.com> <CAGn858RE4p8gez+b0=9PSsZQ=Y1uZANno5V7tqSo=cuqY7AJLA@mail.gmail.com> <BD32CF3D-C6F3-4CF6-A618-C41ED0C4D1CB@cisco.com> <CAGn858SLr4vix18=09gXgsN-VOspBL=qZ2-q6dWyF5b3ASgCYA@mail.gmail.com> <BYAPR11MB25845CFB28F096937486F8D7DAA50@BYAPR11MB2584.namprd11.prod.outlook.com> <CAGn858QOPgXb=-WgWhXETKgEw5v1soo=JsDB+LemOr7G6DKB1A@mail.gmail.com> <9FFC50F3-C5E6-4036-8A4D-29DCE2528B91@alibaba-inc.com> <CAPDqMepJsFPy3Gfh7MC2cJwoywK+YVxfyMw0wZtVyw79r8t6_g@mail.gmail.com> <3FDB1B26-B286-4F3A-ABB9-DACE051F0E5D@alibaba-inc.com>
In-Reply-To: <3FDB1B26-B286-4F3A-ABB9-DACE051F0E5D@alibaba-inc.com>
From: Tom Herbert <tom@quantonium.net>
Date: Thu, 22 Aug 2019 14:11:28 -0700
Message-ID: <CAPDqMert5S2SMBKCynLTcDQE86MAvgad_C28=DGjpmCbid3G3A@mail.gmail.com>
To: "OU, Heidi" <heidi.ou@alibaba-inc.com>
Cc: Vijay Rangarajan <vijayr@arista.com>, "Frank Brockners (fbrockne)" <fbrockne@cisco.com>, "draft-ietf-ippm-ioam-data@ietf.org" <draft-ietf-ippm-ioam-data@ietf.org>, IETF IPPM WG <ippm@ietf.org>, Hugh Holbrook <holbrook@arista.com>, Anoop Ghanwani <Anoop.Ghanwani@dell.com>, Surendra Anubolu <surendra.anubolu@broadcom.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://mailarchive.ietf.org/arch/msg/ippm/dg4m56FH2PUisYb7ES52aej8sIk>
Subject: Re: [ippm] Review on draft-ietf-ippm-ioam-data-06
X-BeenThere: ippm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF IP Performance Metrics Working Group <ippm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ippm>, <mailto:ippm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ippm/>
List-Post: <mailto:ippm@ietf.org>
List-Help: <mailto:ippm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ippm>, <mailto:ippm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Aug 2019 21:11:45 -0000

On Thu, Aug 22, 2019 at 1:51 PM OU, Heidi <heidi.ou@alibaba-inc.com> wrote:
>
> Tom,
>
> Thanks for the reply. Anoop also gave some insight offline on the ASIC restriction.
>
> About adding  IOAM as an IP option, I thought it had been  proposed in
>         https://tools.ietf.org/html/draft-kumar-ippm-ifa-01#page-10
> What prevent us from doing that?

Yeah, IP options would be obstensibly be an obvious choice, but proper
support in middleboxes is notoriously bad. Also, they're limited to
forty bytes. In reality, I doubt no one is seriously considering use
of new IPv4 options. However, IPv6 extension headers are an active
area of development (including the definition of IOAM options). I've
written a draft to allow IPv4 to carry the same HBH extension headers
in IPv4 to bridge the gap between v4 and v6.

Tom

>
> From deployment point of view,  as long as the INT packet can go through the exact same path/queue as the  pre-encap'd packets, we are fine.
>
> Thanks
> - Heidi
>
> On 2019/8/22, 12:41 PM, "Tom Herbert" <tom@quantonium.net> wrote:
>
>     On Thu, Aug 22, 2019 at 12:21 PM OU, Heidi <heidi.ou@alibaba-inc.com> wrote:
>     >
>     > Hi Frank,
>     >
>     >
>     >
>     > I also have a question on the encapsulation: If you can get a new ethertype for IOAM, why not insert IOAM data directly after layer2 MAC?  instead of adding a GRE header for IOAM.
>     >
>     Because, we need a packet format that is compatible with existing
>     network devices. In light of that, GRE is more preferable than using
>     the new Ethertype directly in an Ethernet frame. There will also be
>     similar arguments made for using GRE/IP, and UDP encapsulation over
>     IP, and there was even a proposal to somehow insert the IOAM data
>     immediately after the TCP header and before the TCP data. All of these
>     are attempts to use protocol headers that are thought to be most
>     palatable to intermediate devices and maximize the chances of
>     efficient delivery.
>
>     IMO, all of the aforementioned techniques have some problem or aren't
>     clean (including the GRE solution). The best solution, and most
>     architecturally correct and generic one, is an IOAM option in
>     Hop-by-Hop extension headers.
>
>     Tom
>
>     >
>     >
>     > Thanks
>     >
>     > Heidi
>     >
>     >
>     >
>     > From: Vijay Rangarajan <vijayr@arista.com>
>     > Date: Thursday, August 22, 2019 at 7:22 AM
>     > To: "Frank Brockners (fbrockne)" <fbrockne@cisco.com>
>     > Cc: "Carlos Pignataro (cpignata)" <cpignata@cisco.com>, Jai Kumar <jai.kumar@broadcom.com>, "draft-ietf-ippm-ioam-data@ietf.org" <draft-ietf-ippm-ioam-data@ietf.org>, IETF IPPM WG <ippm@ietf.org>, Hugh Holbrook <holbrook@arista.com>, Anoop Ghanwani <Anoop.Ghanwani@dell.com>, "OU, Heidi" <heidi.ou@alibaba-inc.com>, Surendra Anubolu <surendra.anubolu@broadcom.com>, John Lemon <john.lemon@broadcom.com>
>     > Subject: Re: [ippm] Review on draft-ietf-ippm-ioam-data-06
>     >
>     >
>     >
>     > Hi Frank:
>     >
>     > Thanks, I knew I was missing something.
>     >
>     > So basically what you are saying is - let's say we have a UDP packet, we are just going to stick in the GRE header and IOAM Header and Metadata in-between the original IP and UDP headers?
>     >
>     >
>     >
>     > So, the next protocol in the IOAM Header should indicate the L4 protocol - i.e UDP/TCP?
>     >
>     > Looking at https://datatracker.ietf.org/doc/draft-weis-ippm-ioam-eth/, it actually defines the "Next protocol" in the IOAM header to be an ethertype value?
>     >
>     >
>     >
>     > Thanks,
>     >
>     > Vijay
>     >
>     >
>     >
>     >
>     >
>     > On Thu, Aug 22, 2019 at 6:22 PM Frank Brockners (fbrockne) <fbrockne@cisco.com> wrote:
>     >
>     > Hi Vijay,
>     >
>     >
>     >
>     > note that you don’t necessarily need to “tunnel” – you can just use the GRE header to sequence-in IOAM.
>     >
>     >
>     >
>     > Cheers, Frank
>     >
>     >
>     >
>     > From: Vijay Rangarajan <vijayr@arista.com>
>     > Sent: Donnerstag, 22. August 2019 05:31
>     > To: Carlos Pignataro (cpignata) <cpignata@cisco.com>
>     > Cc: Jai Kumar <jai.kumar@broadcom.com>; draft-ietf-ippm-ioam-data@ietf.org; IETF IPPM WG <ippm@ietf.org>; Frank Brockners (fbrockne) <fbrockne@cisco.com>; Hugh Holbrook <holbrook@arista.com>; Anoop Ghanwani <Anoop.Ghanwani@dell.com>; OU, Heidi <heidi.ou@alibaba-inc.com>; Surendra Anubolu <surendra.anubolu@broadcom.com>; John Lemon <john.lemon@broadcom.com>
>     > Subject: Re: [ippm] Review on draft-ietf-ippm-ioam-data-06
>     >
>     >
>     >
>     > Thanks Carlos, for pointing me to the draft.
>     >
>     >
>     >
>     > Based on my understanding of the two drafts I have the following questions and concerns:
>     >
>     > If I understand correctly, to deploy inband telemetry, we would need to construct GRE tunnels coinciding with the IOAM domain?
>     > GRE typically requires configuration to provision the tunnels. Provisioning and managing these tunnels and keeping these updated as the network grows/shrinks could be a significant overhead.
>     > In order to get the benefit of telemetry, we are imposing a change in forwarding protocol/topology and configuration - which, I feel is not desirable. For example, a customer might have basic L3 routing enabled and the expectation would be for inband telemetry to work seamlessly, without having to revamp the network with GRE tunnels and such. This could be a significant barrier to deployment.
>     > If sampling is used to select packets for performing IOAM encap, is the expectation that only sampled IOAM packets go through GRE encap? Or all data packets?
>     > Due to network nodes inserting the IOAM data, the inner L3/L4 headers keep getting pushed deeper. I would imagine this gets challenging for ASICs to access these fields for hashing/load balancing.
>     > Assuming only a subset of packets in a flow are subject to IOAM (based on sampling), how do we ensure these packets take the same network path as the rest of the packets in the flow?
>     >
>     > Thanks,
>     >
>     > Vijay
>     >
>     >
>     >
>     >
>     >
>     > On Wed, Aug 21, 2019 at 5:04 PM Carlos Pignataro (cpignata) <cpignata@cisco.com> wrote:
>     >
>     > Hello, Vijay,
>     >
>     >
>     >
>     > Please see https://datatracker.ietf.org/doc/draft-weis-ippm-ioam-eth/, and the document this replaces.
>     >
>     >
>     >
>     > Thanks!
>     >
>     > Thumb typed by Carlos Pignataro.
>     >
>     > Excuze typofraphicak errows
>     >
>     >
>     > 2019/08/21 6:35、Vijay Rangarajan <vijayr@arista.com>のメール:
>     >
>     > Hello all:
>     >
>     > Apologise if this has been previously discussed.
>     >
>     > In reading "draft-ietf-ippm-ioam-data-06", I don't see mention of GRE encap. The draft, in fact in Sec 3, says the following - "The in-situ OAM data field can be transported by a variety of transport protocols, including NSH, Segment Routing, Geneve, IPv6, or IPv4.  Specification details for these different transport protocols are outside the scope of this document."
>     >
>     >
>     >
>     > Is there another document, or a description somewhere, that talks about how IOAM is proposed to be carried in GRE? what would be the GRE payload, the GRE protocol type etc?
>     >
>     >
>     >
>     > Thanks,
>     >
>     > Vijay
>     >
>     >
>     >
>     >
>     >
>     > On Wed, Aug 21, 2019 at 7:52 AM Jai Kumar <jai.kumar@broadcom.com> wrote:
>     >
>     > Hello Frank,
>     >
>     >
>     >
>     > This is in context of our conversation at IETF105. My goal is to provide input and improve current IOAM data draft with the learnings we had with IFA deployment.
>     >
>     > This feedback is based on various customer interactions and concerns raised by them wrt IOAM. Each feedback is a longer topic and I am starting this thread as a summary email. This is just highlighting the issues and not yet proposing any solution.
>     >
>     >
>     >
>     >
>     >
>     > Feedback 1:
>     >
>     > Section 4.2..1 Pre-allocated and Incremental Trace Options
>     >
>     > Pre-allocated and incremental trace option is 8Bytes long. This can be easily reduced to 4Bytes.
>     >
>     > There is a feedback that pre-allocated option is really not needed and either be removed or made optional.
>     >
>     > Given that deployments are sensitive to the IOAM overhead (specially in 5G deployments), it’s a 50% fixed overhead savings on a per packet basis.
>     >
>     >
>     >
>     >
>     >
>     > Feedback 2:
>     >
>     > Section 4.1 IOAM Namespaces
>     >
>     > Namespaces should be treated as templates (similar to IPFIX template record formats). This is more flexible way of enumerating data. 64K namespace id is a very large namespace and can be reduced to 64 IANA specified name spaces. Separate private name space can be allowed instead of interleaving of opaque data in the IANA allocated name space as suggested in the current draft “opaque state snapshot”.
>     >
>     > https://tools.ietf.org/html/rfc7011#section-3.4
>     >
>     >
>     >
>     > Feedback 3:
>     >
>     > Section 4.2.1 Pre-allocated and Incremental Trace Options
>     >
>     > IOAM-Trace-Type:  A 24-bit identifier which specifies which data
>     >
>     >       types are used in this node data list.
>     >
>     > This is the most contentious of all. In the current proposal, as new data fields are added, there is a corresponding trace type bit need in the header. This essentially means that all possible data fields need to be enumerated. Given that we there are 64K names spaces allowed, I don’t see how we can fit all possible data fields in this 24bit vector. I know there was a suggestion of keeping last bit as an extension bit but it is still scalable and/or easy to implement in hardware. Besides this the data fields are not annotated/encoded with the data type, something like in IPFIX https://tools.ietf.org/html/rfc7011#section-6.1
>     >
>     >
>     >
>     > Feedback 4:
>     >
>     > There is no version field in the data header and this will make interoperability challenging. Standard will evolve and headers bit definition and/or trace type will change and without version field HW will not be able to correctly handle the IOAM data headers.
>     >
>     >
>     >
>     > Feedback 5:
>     >
>     > Handling of TCP/UDP traffic using GRE encap is not acceptable. Here are some of the issues I can think of
>     >
>     > GRE encaped IOAM packets will traverse a different network path then the original packet
>     > Not all packets can be GRE encaped to avoid the previous problem, due to wastage of network bandwidth (typically sampled traffic is used for IOAM). What about native GRE traffic, will it get further encaped in another GRE tunnel and so forth.
>     > IP header protocol will point to GRE IP proto and IOAM ethertype (pending allocation by IEEE) need to be read from the GRE header to detect an IOAM packet. This means parsing performance penalty for all regular GRE (non IOAM) traffic.
>     >
>     >
>     >
>     > Thanks,
>     >
>     > -Jai
>     >
>     >
>     >
>     > _______________________________________________
>     > ippm mailing list
>     > ippm@ietf.org
>     > https://www.ietf.org/mailman/listinfo/ippm
>
>
>