[nvo3] Using Geneve for OAM

[[e-mail] (Dale R. Worley)]

I'm new to this work, and as an exercise, I've been looking into using
Geneve to carry OAM information.  Doing so suggests that certain
adjustments to Geneve would make it better for OAM usage.  I'd be glad
to hear feedback about any of these ideas.

The design principles I am following are:

- Here, "OAM" is restricted to monitoring ordinary data packets; such
  OAM tasks as telnetting into a router are handled by ordinary packet
  streams addressed in the proper way.

- We are interested in both "active" OAM (where the monitoring packet
  does not carry non-OAM data) as well as "hybrid" or "in-situ" OAM
  (where the monitoring packet carries both OAM information as well as
  non-OAM data).

- Using Geneve as a "universal" encapsulation protocol, where it can
  be inserted into the sequence of headers of a packet at any point, to
  carry both OAM and non-OAM annotations for the remaining packet
  contents.  (I am particularly fond of this concept.)

- Support "fast path" processing for Geneve headers that conform to a
  pre-established profile that specifies the sequence of option types
  and lengths.

I am using the "IOAM" proposal (draft-ietf-ippm-ioam-data-02) for the
semantics and syntax of the OAM data, basically because it seems to be
fairly complete, and I have little knowledge about what the OAM data
should be.  The IOAM proposal reduces the OAM information to three
alternative data blocks (tracing, proof of transit, and edge-to-edge),
each of which is configurable in numerous ways.
draft-brockners-nvo3-ioam-geneve-00 proposes that each of these data
blocks can be included in a Geneve header under a particular
class/type code.

(The major lack in the IOAM proposal is carriage of echo request/reply
messages.  draft-ooamdt-rtgwg-demand-cc-cv seems to be a promising
approach to this OAM message type.)

"Universality" for Geneve is accomplished in part by extending the
allowed underlying protocols (preceding header types).  This involves
establishing an Ethertype for Geneve (so that Geneve can be
encapsulated directly in Ethernet) and establishing an IP
protocol/header value (so that Geneve can be encapsulated in
IPv4/IPv6).  Similarly, Geneve needs to be extended so that its
"protocol type" field can carry an IP protocol/header value (so that
Geneve can encapsulate IETF layer 4 protocols) and a "no encapsulated
data" value.  IP protocol/header values can be encoded by setting the
high octet of the protocol type field to 0x00 and the low octet to the
protocol/header value.  (Ethertypes have a high octet of 0x06 or
higher, so this does not conflict with the present specification of
Geneve.)  Packets for "active" OAM, which contain no overlay data,
indicate "no encapsulated data" with the value 0x00 0x3B, since
protocol/header value 59 means "no next header".

The second word of the fixed part of a Geneve header carries the
virtual network identifier, which may not be needed.  It seems to me
to be a useful feature to define one of the Geneve header flag bits to
mean "short header"; if it is set, the second word of the fixed part
is absent.  With this extension, a minimal Geneve header carrying IOAM
looks like this:

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ <--
 |Ver|  Opt Len  |O|C|S|  Rsvd.  |          Protocol Type        |  Geneve fixed
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ <--
 |  Option Class = IOAM_Trace    |C| IOAM-Trace  |R|R|R| Length  |  Geneve option
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ <--
 |         IOAM-Trace-Type       |NodeLen|  Flags  | Octets-left |  IOAM fixed
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ <--
 |                                                               |  IOAM data
 ~                        node data list [0]                     ~  space
 |                                                               |   
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ <--

This format is well-versionized, with the Geneve version in the first
two bits of the Geneve fixed part and the IOAM version implied by the
option class/type value.  This requires only 2 words (8 octets) in
addition to the IOAM fixed and data space.  If a Geneve header was
needed at this point in the header stack for other purposes, the
overhead of adding OAM is only 1 word.

As is discussed in draft-brockners-nvo3-ioam-geneve-00 section 4,
Geneve headers and options are rather short for IOAM tracing:  the
maximum data part of an option is (2^5-1)*4 = 124 octets, and the
maximum option part of a header is (2^6-1)*4 = 252 octets.  It's worth
considering extending the length fields in Geneve to support longer
options.  Consider this format:  The flag bits in the header fixed
part is reduced to 1 bit (short header), extending the length field to
13 bits.  The reserved bits in the option fixed part are added to the
length field, extending it to 8 bits.

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ <--
 |Ver|  Opt Len                |S|          Protocol Type        |  Geneve fixed
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ <--
 |  Option Class = IOAM_Trace    |C| IOAM-Trace  |       Length  |  Geneve option
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ <--
 |         IOAM-Trace-Type       |NodeLen|  Flags  | Octets-left |  IOAM fixed
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ <--
 |                                                               |  IOAM data
 ~                        node data list [0]                     ~  space
 |                                                               |   
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ <--

The maximum data part of an option is now (2^8-1)*4 = 1,020 octets and
the maximum option part of a header is now (2^13-1)*4 = 32,764 octets.
In the case mentioned in draft-brockners-nvo3-ioam-geneve-00, where
each visited node adds 20 octets of tracing information, the IOAM data
space could accommodate floor((1020 - 4) / 20) = 50 hops of
information.

This revised format omits the O and C bits from the header fixed
part.  As far as I can tell -- and I may be wrong about this -- this
causes no inconvenience, as the Geneve header is only examined by
nodes which are endpoints for all of the encapsulating protocols, that
is, for the addresses carried in the preceding headers in the packet.
And in that case, the node is required to examine all of the options
in the Geneve header, in case it is required to take some action based
on them.  In that case, the C bit in the fixed part carries no
information that the node cannot determine by examining all of the
options, which it will do anyway.  Similarly, the O bit for
identifying OAM information can carry no information that is not
carried elsewhere in the Geneve header.

One aspect of Geneve processing that I am trying to support is "fast
path" processing.  The concept is that the node is configured for a
certain "profile" of Geneve header -- a particular sequence of Geneve
option types and lengths -- and that the node should be able to
quickly identify packets that conform to the profile and divert them
to a preconfigured processing path, leaving non-conforming packets to
be processed by a more general and slower mechanism.  As long as the
sequence of options and their lengths are fully specified by the
profile, the profile conformance test is "TCAM compatible", that is,
can be done by examining whether certain bits at certain offsets from
the start of the Geneve header have certain values.

If more brevity is required, we can eliminate the IOAM fixed part by
incorporating the 16 bit IOAM-Trace-Type into the Geneve option
class/type -- allocate a block of 2^16 of the 2^23 class/type
identifiers to specify IOAM-Trace-Types:

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ <--
 |Ver|  Opt Len                |S|          Protocol Type        |  Geneve fixed
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ <--
 |  Class      | Trace-Type      |C| Trace-Type  |       Length  |  Geneve option
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ <--
 |                                                               |  IOAM data
 ~                        node data list [0]                     ~  space
 |                                                               |   
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ <--

This uses 2^-7 of the option/class number space to support IOAM-Trace,
but it might be worth it to reduce the overhead for IOAM trace to 1
word (4 octets), or 0 words if a Geneve header has to be inserted for
other purposes already.

This approach does require an alternative way to describe how many
entries in the IOAM data are filled in, since the "Octets-left" field
of the IOAM fixed part is lost.  One way to represent this is
to not fill the entries in sequentially as the packet passes through
nodes, but rather have each node insert its information into the
beginning of the data space, and shift the entries previously present
in the data space later:

  +----------+            +----------+            +----------+
  | empty    |		  | entry 1  |            | entry 2  |
  +----------+		  +----------+            +----------+
  | empty    |		  | empty    |            | entry 1  |
  +----------+   ----->	  +----------+   ----->   +----------+
  | empty    |		  | empty    |            | empty    |
  +----------+		  +----------+            +----------+
  | empty    |		  | empty    |            | empty    |
  +----------+		  +----------+            +----------+
  | empty    |		  | empty    |            | empty    |
  +----------+		  +----------+            +----------+

In order for the final node to determine how many entries had been
filled in, it helps to have a defined empty value.
draft-ietf-ippm-ioam-data-02 suggests that the value 0xFFFFFFFF
represents an empty word, although there are a few situations where
that value could be inserted as a non-null value (e.g., in
timestamps), and some provision would be needed to handle that.

Overflowing of the data area could be detected by making the data area
not a multiple of the length of an entry, but one word longer than a
multiplea.  If the final word has a non-null value, then the final node
can tell that some information was lost.

Dale