Re: [ippm] Review on draft-ietf-ippm-ioam-data-06

Vijay Rangarajan <> Thu, 22 August 2019 12:31 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 18724120848 for <>; Thu, 22 Aug 2019 05:31:17 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id MkTDxc4EG-MB for <>; Thu, 22 Aug 2019 05:31:13 -0700 (PDT)
Received: from ( [IPv6:2607:f8b0:4864:20::e2a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 8F31F120820 for <>; Thu, 22 Aug 2019 05:31:13 -0700 (PDT)
Received: by with SMTP id q16so3718779vsm.2 for <>; Thu, 22 Aug 2019 05:31:13 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=googlenew; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=XY/M2b4pKBI+IW0+6+tm1GwHL4sxiRP6+klVrGqH2Po=; b=GYQIvtXL5czWzZmjF8sAUu23OfYOu9/AmIz6ssATVpI62dUCqE36Vm3Z71DGy7z3zE zxbPsJcFHggvIOqLMHuXmOVPylgQsR6bynABnefspYsjiHS60Z263tawGmER71VbVcRN N2vBUkUgX9jpKcCt8cvsp6y8EmJ40DnMLGr3MyDdVC2N/6N9rTOO0JmbdvXdjS13VYVY I1BNXgy21rrg+f029orVba6BVT6EeBojsJ5JxjNbW+WCN5m2kk/hpAJrc37MNSgynybP sxc9YWRg0phhaqPNhqWch9oOE7JjGQbTjmuDObpfsWJhxaSkHTbhIpKvD10JgRle4p8i tZdA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=XY/M2b4pKBI+IW0+6+tm1GwHL4sxiRP6+klVrGqH2Po=; b=sTk9SqXvbppRsEtGaoNfWlR87yUBeAFqH5C9fWLrEt6+RGKnDFssv0DJFg4Q5Do9pZ lnT7wHo4FPhuEmWNfa822RDdlnk3ClBDU2ArdcYNkAlMUDCtLWL4ol74M+IT5W3WBXde MJgKAD6EAQqJaO2+nLjjk8+4z6shHIhvYqKbUZxliATypZYmMpw4MxgvuSTpFIFlBtvC irpKbTtJEb7AqysMPP1fVhjDb/GU5paomgcbEdYg5BUfbK2YSRVtZY9Od53GCUj51jCN SesPsrQTX/0WJvpQy/1fmYWxZ0aMBd7pZvqZ1dSuUWgVBfxGYBcsFcbAln6W9lDLVIY3 ViIg==
X-Gm-Message-State: APjAAAWjEflCghSeCFUw/7cIqrO2nlLGg7VEZQw6SznQEiJUSmsY4FHG Cm9Qu6ourLVrchVXqjyshEUmEUQclPTrGi8riX5JIg==
X-Google-Smtp-Source: APXvYqyHnMReoDnBGUb9+uU++svc319lTIBwPIl3KQdwwj33ZTSampmTklpHr55Dmy5re7C1X0uLjLRPVUnayPsMgoA=
X-Received: by 2002:a67:d309:: with SMTP id a9mr24165573vsj.213.1566477072442; Thu, 22 Aug 2019 05:31:12 -0700 (PDT)
MIME-Version: 1.0
References: <> <> <>
In-Reply-To: <>
From: Vijay Rangarajan <>
Date: Thu, 22 Aug 2019 18:01:00 +0530
Message-ID: <>
To: "Carlos Pignataro (cpignata)" <>
Cc: Jai Kumar <>, "" <>, IETF IPPM WG <>, "Frank Brockners (fbrockne)" <>, Hugh Holbrook <>, Anoop Ghanwani <>, "OU, Heidi" <>, Surendra Anubolu <>, John Lemon <>
Content-Type: multipart/alternative; boundary="00000000000073d9730590b3dd21"
Archived-At: <>
X-Mailman-Approved-At: Thu, 22 Aug 2019 10:06:26 -0700
Subject: Re: [ippm] Review on draft-ietf-ippm-ioam-data-06
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF IP Performance Metrics Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 22 Aug 2019 12:31:17 -0000

Thanks Carlos, for pointing me to the draft.

Based on my understanding of the two drafts I have the following questions
and concerns:

   1. If I understand correctly, to deploy inband telemetry, we would need
   to construct GRE tunnels coinciding with the IOAM domain?
   2. GRE typically requires configuration to provision the tunnels.
   Provisioning and managing these tunnels and keeping these updated as the
   network grows/shrinks could be a significant overhead.
   3. In order to get the benefit of telemetry, we are imposing a change in
   forwarding protocol/topology and configuration - which, I feel is not
   desirable. For example, a customer might have basic L3 routing enabled and
   the expectation would be for inband telemetry to work seamlessly, without
   having to revamp the network with GRE tunnels and such. This could be a
   significant barrier to deployment.
   4. If sampling is used to select packets for performing IOAM encap, is
   the expectation that only sampled IOAM packets go through GRE encap? Or all
   data packets?
   5. Due to network nodes inserting the IOAM data, the inner L3/L4 headers
   keep getting pushed deeper. I would imagine this gets challenging for ASICs
   to access these fields for hashing/load balancing.
   6. Assuming only a subset of packets in a flow are subject to IOAM
   (based on sampling), how do we ensure these packets take the same network
   path as the rest of the packets in the flow?


On Wed, Aug 21, 2019 at 5:04 PM Carlos Pignataro (cpignata) <> wrote:

> Hello, Vijay,
> Please see,
> and the document this replaces.
> Thanks!
> Thumb typed by Carlos Pignataro.
> Excuze typofraphicak errows
> 2019/08/21 6:35、Vijay Rangarajan <>のメールt;のメール:
> Hello all:
> Apologise if this has been previously discussed.
> In reading "draft-ietf-ippm-ioam-data-06", I don't see mention of GRE
> encap. The draft, in fact in Sec 3, says the following - "The in-situ OAM
> data field can be transported by a variety of transport protocols,
> including NSH, Segment Routing, Geneve, IPv6, or IPv4.  Specification
> details for these different transport protocols are outside the scope of
> this document."
> Is there another document, or a description somewhere, that talks about
> how IOAM is proposed to be carried in GRE? what would be the GRE payload,
> the GRE protocol type etc?
> Thanks,
> Vijay
> On Wed, Aug 21, 2019 at 7:52 AM Jai Kumar <> wrote:
>> Hello Frank,
>> This is in context of our conversation at IETF105. My goal is to provide
>> input and improve current IOAM data draft with the learnings we had with
>> IFA deployment.
>> This feedback is based on various customer interactions and concerns
>> raised by them wrt IOAM. Each feedback is a longer topic and I am starting
>> this thread as a summary email. This is just highlighting the issues and
>> not yet proposing any solution.
>> Feedback 1:
>> Section 4.2.1 Pre-allocated and Incremental Trace Options
>> Pre-allocated and incremental trace option is 8Bytes long. This can be
>> easily reduced to 4Bytes.
>> There is a feedback that pre-allocated option is really not needed and
>> either be removed or made optional.
>> Given that deployments are sensitive to the IOAM overhead (specially in
>> 5G deployments), it’s a 50% fixed overhead savings on a per packet basis.
>> Feedback 2:
>> Section 4.1 IOAM Namespaces
>> Namespaces should be treated as templates (similar to IPFIX template
>> record formats). This is more flexible way of enumerating data. 64K
>> namespace id is a very large namespace and can be reduced to 64 IANA
>> specified name spaces. Separate private name space can be allowed instead
>> of interleaving of opaque data in the IANA allocated name space as
>> suggested in the current draft “opaque state snapshot”.
>> Feedback 3:
>> Section 4.2.1 Pre-allocated and Incremental Trace Options
>> IOAM-Trace-Type:  A 24-bit identifier which specifies which data
>>       types are used in this node data list.
>> This is the most contentious of all. In the current proposal, as new data
>> fields are added, there is a corresponding trace type bit need in the
>> header. This essentially means that all possible data fields need to be
>> enumerated. Given that we there are 64K names spaces allowed, I don’t see
>> how we can fit all possible data fields in this 24bit vector. I know there
>> was a suggestion of keeping last bit as an extension bit but it is still
>> scalable and/or easy to implement in hardware. Besides this the data fields
>> are not annotated/encoded with the data type, something like in IPFIX
>> Feedback 4:
>> There is no version field in the data header and this will make
>> interoperability challenging. Standard will evolve and headers bit
>> definition and/or trace type will change and without version field HW will
>> not be able to correctly handle the IOAM data headers.
>> Feedback 5:
>> Handling of TCP/UDP traffic using GRE encap is not acceptable. Here are
>> some of the issues I can think of
>>    - GRE encaped IOAM packets will traverse a different network path
>>    then the original packet
>>    - Not all packets can be GRE encaped to avoid the previous problem,
>>    due to wastage of network bandwidth (typically sampled traffic is used for
>>    IOAM). What about native GRE traffic, will it get further encaped in
>>    another GRE tunnel and so forth.
>>    - IP header protocol will point to GRE IP proto and IOAM ethertype
>>    (pending allocation by IEEE) need to be read from the GRE header to detect
>>    an IOAM packet. This means parsing performance penalty for all regular GRE
>>    (non IOAM) traffic.
>> Thanks,
>> -Jai