[lisp] Comments to draft-ietf-lisp-introduction-02

[Dino Farinacci <[e-mail]>]

I sent these comments last week to Noel since he gave me a sneak preview of draft-ietf-lisp-introduction-02. With his permission, I have included them below.

I believe he plans to respond publicly to the comments. Thanks Noel!

Dino


Begin forwarded message:

> From: Dino Farinacci <[e-mail]>
> Subject: Re: Intro and Perspective documents
> Date: August 3, 2013 4:40:46 AM PDT
> To: Noel Chiappa <[e-mail]>
> Cc: Terry Manderson <[e-mail]>, "[e-mail] Halpern" <[e-mail]>, Dino Farinacci <[e-mail]>
> 
>>   http://ana-3.lcs.mit.edu/~jnc/tech/lisp/LISP_Intro_iv2.txt
> 
> Noel, here are my comments. Your text comes first and is indented and my comments follow. Let me know if I should post this to the list and I will.
> 
> General comments:
> 
> (1) There are many places where you have parenthetical comments. In most of those cases, the text I think is important so I urged you remove the parens but keep the text.
> 
>>    5.  Initial Applications
>>      5.1.  Provider Independence
>>      5.2.  Multi-Homing
>>      5.3.  Traffic Engineering
>>      5.4.  Routing
>>      5.5.  Mobility
>>      5.6.  IP Version Reciprocal Traversal
>>      5.7.  Local Uses
> 
> I think multicast should move from section 13 to here. It is pretty fundamental as all the use-cases you list above. Especially since there is an RFC for LISP-Multicast where some listed above have no RFC (yet).
> 
>>    9.  xTRs
> 
> Could you name this "Data Plane - XTRs".
> 
>>      9.1.  When to Encapsulate
>>      9.2.  UDP Encapsulation Details
>>      9.3.  Header Control Channel
>>        9.3.1.  Mapping Versioning
>>        9.3.2.  Echo Nonces
>>        9.3.3.  Instances
>>      9.4.  Probing
>>      9.5.  Mapping Lifetimes and Timeouts
>>      9.6.  Security of Mapping Lookups
>>      9.7.  Mapping Gleaning in ETRs
>>      9.8.  Fragmentation
>>    10. The Mapping System
> 
> And can you name this "Control Plane - The Mapping System".
> 
>>      10.1. The Mapping System Interface
> 
> Just a suggestion but maybe not necessary but do you like the term "Mapping System API"? An API that does not change if the Mapping Database transport system would change.
> 
>>        10.1.1. Map-Request Messages
>>        10.1.2. Map-Reply Messages
>>        10.1.3. Map-Register and Map-Notify Messages
> 
> Just an FYI we have Map-Notify-Ack messages as well.
> 
>>      10.2. The DDT Indexing Sub-system
>>        10.2.1. Map-Referral Messages
> 
> I am really glad that DDT is listed here but does it imply a precedent or a favoring of one transport system versus others? Especially since DDT is not yet RFC but ALT is. Is there a way to talk about the indexing system without mentioning any specific one?
> 
> I don't have a strong opinion here because I want to be practical and I think describing the Map-Referral message is important. The fact that you mention all LISP messages in one place is goodness.
> 
> Having said that, a mention for the Info-Request and Info-Reply for NAT-traversal will be important as well.
> 
>>      12.4. Neighbour Liveness
>>      12.5. Neighbour Reachability
> 
> We have never used the term Neighbour. Can you use "RLOC Liveness" and "RLOC Reachability"?
> 
>> 1.  Prefatory Note
>> 
>>    {{This section can probably be edited down somewhat.}}
> 
> Let's see if I can help.
> 
>>    This document is the first of a pair which, together, form what one
>>    would think of as the 'architecture document' for LISP (the
>>    'Location-Identity Separation Protocol').  Much of what would
>>    normally be in an architecture document (e.g. the architectural
>>    design principles used in LISP, and the design considerations behind
>>    various components and aspects of the LISP system) is in the second
>>    document, the 'Architectural Perspective on LISP' document.
>> 
>>    This 'Architectural Introduction' document is primarily intended for
>>    those who don't know all about LISP, and want to start learning about
> 
> How about "... intended for newcomers to LISP, ...".
> 
>>    it.  It is intended to both be easy to follow, and also to give the
>>    reader a choice as to how much they wish to know about LISP.  Reading
>>    only the first part(s) of the document will give a good high-level
>>    view of the system; reading the complete document should provide a
>>    fairly detailed understanding of the entire system.
>> 
>>    This goal explains why the document has a somewhat unusual structure.
>>    It is not a typical reference document, where all the content on a
>>    particular topic is grouped in one place.  (That role is filled by
>>    the various protocol specifications.)  Instead, it is structured as a
>>    series of phases, each covering the entire system, but with
>>    increasing detail.
> 
> I would remove this entire paragraph. It is not necessary to be subjective about the structure. 
> 
>>    It starts with a very high-level view of the entire system, to
>>    provide readers with a mental framework to help understand the more
>>    detailed material which follows.  A second pass over the whole system
> 
> Could you change to "... detailed material which are contained in the protocol references. ...".
> 
>>    then goes into more detail.  Finally, individual sub-systems are
>>    covered in still deeper detail.
>> 
>>    The intent is two-fold: first, the multiple passes over the entire
>>    system, each one going into more detail, are intended to ease
>>    understanding; second, people can simply stop reading when they have
>>    a detailed-enough understanding for their purposes.
> 
> I think you can remove the above paragraph. I think it just repeats what you said above.
> 
>>    People who just want to get an idea of how LISP works might only read
>>    the first two parts; they can stop reading either just before, or
>>    just after, the Section 7 Section.  People people who are going to go
>>    on and read all the protocol specifications (perhaps to implement
>>    LISP) would need/want to read the entire document.
> 
> I would remove this paragraph as well. People can decide for themselves how much to read. 
> 
>>    Note: This document is a descriptive document, not a protocol
>>    specification.  Should it differ in any detail from any of the LISP
>>    protocol specification documents, they take precedence for the actual
>>    operation of the protocol.
> 
> This is important to state. Thanks.
> 
>> 
>> 2.  Background
>> 
>>    [[B0: Things inside '[[]]' are editorial comments.  For now they show
>>    up - when we go to produce the ID, I will make them invisible.]]
>> 
>>    It has gradually been realized in the networking community that
>>    networks (especially large networks) should deal quite separately
>>    with the identity and location of a node - basically, 'who' a node
>>    is, and 'where' it is.  (A more detailed history of this evolution is
>>    in Appendix B.1.)
> 
> Remove the parens and just put in an reference to the B.1 anchor.
> 
>>    At the moment, in both IPv4 and IPv6, IP addresses indicate both
>>    where the named device is, as well as identify it for purposes of
>>    end-end communication.  However, the separation of location and
>>    identity is a step which has recently been identified by the IRTF as
>>    a critically necessary evolutionary architectural step for the
>>    Internet.  [RFC6115]
>> 
>>    The on-going LISP project is an attempt to provide a viable path
>>    towards this separation.  (A brief history of the LISP project can be
>>    found in Appendix B.2.)  As an add-on to a (large) existing system,
> 
> Remove the parens but keep the text.
> 
>>    it has had to make certain compromises.  (For a good example, see
>>    [Perspective], Section "Residual Location Functionality in EIDs".)
>>    However, if it reaches near-ubiquitous deployment, it will have two
>>    important consequences.
> 
> I would again remove the parens but keep the text.
> 
>> 3.1.  Economics
> 
> I thought this section was great!
> 
>> 3.3.  'Self-Deployment'
>> 
>>    LISP has deliberately employed a rather different deployment model,
>>    which we might call 'self-deployment' (for want of a better term); it
>>    does not require a huge push to get it deployed, rather, it is hoped
>>    that once people see it and realize they can easily make good use of
>>    it _on their own_ (i.e. without requiring adoption by others), it
>>    will 'deploy itself' (hence the name of the approach).
> 
> This is great text. I am glad you have it.
> 
>>    architecture.  Faailing to clearly recognize both interfaces and
> 
> Change to "Failing".
> 
>>    communication stacks as distinctly separate classes of things is
>>    another failing of the existing Internet architecture (again, one
>>    inherited from the previous generation of networking).
>> 
>>    A novelty in LISP is that it uses existing IPvN addresses (initially,
>>    at least) for both of these kinds of names, thereby minimizing the
>>    deployment cost, as well as providing the ability to easily interact
>>    with unmodified hosts and routers.
> 
> Not sure if this is the best place to mention it, but do you want to say anything about how EIDs and RLOCs can be of any address-family and have multi-tuple encodings?
> 
>> 4.2.  Basic Functionality
>> 
>>    The basic operation of LISP, as it currently stands, is that LISP
>>    augmented packet switches [[O2: do we need a new term for these
>>    things, or is 'router' OK, or should we use an existing, but
>>    uncomittal term, like 'device' - I guess I will call them 'LISP
>>    devices' to start with, and 'xTRs' once I have introduced that term]]
> 
> I think "LISP router" is okay, well-known and obvious. But if you want to generalize to refer to LISP-MNs as well then we should call it a "LISP node". And you should say that a LISP node is a router, switch or LISP-MN that supports LISP.
> 
> The term "device" could lend itself to not make people think that the LISP node may not be virtual, where in practice, it is physical and virtual.
> 
>>    The lookup is performed through a 'mapping system', [[O3: the
>>    existing LISP documents aren't wholly consistend on this name, many
>>    call it the mapping database, but I want to stick with 'mapping
>>    system', to emphasize its unity in its own right]] which is the heart
>>    of LISP: it is a distributed directory of mappings from EIDs to
>>    RLOCS.  The destination RLOC will be (initially at least) the address
>>    of the LISP device near the ultimate destination (the Egress Tunnel
>>    Router, or 'ETR').
> 
> That is fine, but right here call it "... through a 'mapping database' or 'mapping system' ..." and then just use 'mapping system' throughout the document.
> 
>>    {{Is it worth distinguishing between 'mapping' and 'binding'?  Should
>>    the document pick one term, and stick with it?}}
> 
> I don't think there is a difference. So let's stick with mapping so we don't obsolete the text in the RFCs.
> 
>>    Return traffic is handled similarly, often (depending on the
>>    network's configuration) with the original ITR and ETR switching
>>    roles.  The ETR and ITR functionality is usually co-located in a
>>    single device; these are normally denominated as 'xTRs'.
>> 
>>    [[O4: Look at LISP ID to see how it describes this process, see if
>>    their description is any better.]]
> 
> I think the section is fine the way it is.
> 
>> 4.3.  Mapping from EIDs to RLOCs
>> 
>>    The mappings from EIDs to RLOCs are provided by a distributed (and
>>    potentially replicated) database, the mapping database, which is the
>>    heart of LISP.  [[O5: Is that what we want to call it, the 'mapping
>>    database'?  Check other LISP IDs to see if there is a standard
>>    name.]]
> 
> Can you change to ".. which is the heart of the LISP control-plane.".
> 
>>    Mappings are requested on need, not (generally) pre-loaded; in other
>>    words, mapping are distributed via a 'pull' mechanism.  Once obtained
> 
> Well pre-loading could be done by the implementation so you should probably not make a reference to it. And for NAT-travesal, once you get the address of an RTR, you install a default map-cache entry. That is a preload because the cache is repopulated before any packet arrives.
> 
>> 4.4.  Interworking With Non-LISP-Capable Endpoints
>> 
>>    The capability for 'easy' interoperation between nodes using LISP,
>>    and existing non-LISP-using hosts (often called 'legacy' hosts) or
>>    sites (where 'site' is usually taken to mean a collection of hosts,
>>    routers and networks under a single administrative control), is
>>    clearly crucial.
> 
> I think this paragraph needs rephrasing because it came across to me that there are LISP-hosts and legacy hosts that don't run LISP. But the fact of the matter is there are hosts in LISP sites that do not run LISP. So just say there are LISP sites where addresses are in the mapping database and there are non-LISP sites where addresses are advertised to the underlying routing system.
> 
>>    To allow such interoperation, a number of mechanisms have been
>>    designed.  This multiplicity is in part because different mechanisms
>>    have different advantages and disadvantages (so that no single
>>    mechanism is optimal for all cases), but also because with limited
>>    field experience, it is not clear which (if any) approach will be
>>    preferable.  [[O6: Is this true?  Why do we have several mechanisms,
>>    then?]]
> 
> Can you change to "... multiplicity is in part because design tradeoffs are allowing a choice to be made when more field experience is obtained".
> 
>>    To provide a brief overview, it is definitely understood that LISP
>>    needs to be highly _securable_, especially in the long term; over
>>    time, the attacks mounted by 'bad guys' are becoming more and more
>>    sophisticated.  So LISP, like DNS, needs to be _capable_ of providing
>>    'the very best' security there is.
> 
> I would add "... security there is, at a reasonable cost.".
> 
>>    To accomplish these divergent goals, the approach taken is to
>>    thorougly analyze what LISP needs for security, and then design, in
>>    detail, a scheme for providing that security.  Then, steps can be
>>    taken to ensure that the appropriate 'hooks' (such as packet fields)
>>    are included at an early stage, when doing so is still easy.  Later
>>    on, the design can be fully specified, implemented, and deployed.
> 
> This is making it sound like we have no security and it is to be started. You should say that an initial form of security is placed in the API to sign and secure Map-Replies as well as to validate Map-Referrals in DDT. We should say that this amount of security is adequate for the start of deployment and experience will tell us if more mechanism will be required.
> 
> What do you think?
> 
>>    TE is, fundamentally, a routing issue.  However, the current Internet
>>    routing architecture, which is basically the Baran design of fifty
>>    years ago [Baran] (a single large, distributed computationa), is ill-
> 
> Change "computationa" to "computation".
> 
>> 5.5.  Mobility
>> 
>>    Mobility is yet another place where separation of location and
>>    identity is obviously a key part of a clean, efficient and high-
>>    functionality solution.  Considerable experimentation has been
>>    completed on doing mobility with LISP.
>> 
>>    [[A2: What else to say?]]
> 
> You should say that sessions can survive across moves as well as better packet stretch compared to traditional schemes.
> 
>> 
>>    Among the latter class, non-Internet applications which have no
>>    analog on the Internet, are the following example applications:
>>    virtual machine mobility in data centers; other non-IP EID types such
>>    as local network MAC addresses, or application specific data.
>> 
>>    [[A6: What about recursive tunneling?  Do we need an 'advanced usage
>>    cases' section?]]
> 
> Just make a reference to the main RFC 6830.
> 
> Also in section 5, you should include a multicast section, indicating an opportunity to encapsulate multicast in unicast. And it allows for group addressing to be local to the sites attached to the core. So there is less group allocation coordination among the core and the sites.
> 
> This is also true for encapsulating one address-family in another. So that could use a mention somewhere in the applications section.
> 
>>    [[S0: There seems to be a 3-level introduction of the system: the
>>    part at the top has a really high-level description of how it works;
>>    this section gives more detail on major functional units; and then
>>    later on it seems like we have a separate section for each major
>>    functional sub-system, which goes into full detail.  Is this too many
>>    levels - should we just have 'overall intro' and then 'detailed look
>>    at each major functional sub-system'?  Or should this section, after
>>    the _very_ high-level overview previously, spend a page or so
>>    describing what the major functional sub-systems are?  The thing is
>>    that there aren't that many - just xTRs and the mapping system.]]
> 
> Have a very short section and use this wonderful picture that Alberto drew up:
> 
> 
> 
> You'll have to draw it in ascii.
> 
>>    [[S1: Later: I've been working on the arrangement of material, and I
>>    do think it will be useful to have this intermediate level.  My
>>    reasoning is that on every 'pass' across LISP, I drill deeper into
>>    the details.  I think this is a better approach that going straight
>>    into the fine detail of, say, the operation of a Map-Resolver - many
>>    of the details of which might make not make sense without a better
>>    overall sense of how the entire system works.  So section 3 is the
>>    very high level look at LISP, this takes another pass, going a little
>>    deeper, as a way to set up for sections 8-9, which contain all sorts
>>    of low-level engineering detail (after a setup which explains the
>>    engineering design philosophy, to help people understand why
>>    particular engineering choices were made.]]
> 
> Agree. Just try not repeating yourself.
> 
>>    The data plane functions in ITRs include deciding which packets need
>>    to be given LISP processing (since packets to non-LISP hosts may be
>>    sent 'vanilla'); i.e. looking up the mapping; encapsulating the
>>    packet; and sending it to the ETR.  This encapsulation is done using
>>    UDP [RFC768] (for reasons to be explained below, in Section 9.2),
>>    along with an additional IPvN header (to hold the source and
>>    destination RLOCs).  To the extent that traffic engineering features
>>    are in use for a particular EID, the ITRs implement them as well.
>> 
>>    In the ETR, the data plane simply unwraps the packets, and forwards
>>    the now-normal packets to the ultimate destination.
>> 
>>    [[S2: Anything else?]]
> 
> Well, possibly doing some control functions are storing some state. Like echo-nonces, or adding to counters for per RLOC/per-EID packet counters. 
> 
>>    Control plane functions in ITRs include: asking for {EID->RLOC}
>>    mappings via Map-Request control messages; handling the returning
>>    Map-Replies which contain the requested information; managing the
>>    local cache of mappings; checking for the reachability and liveness
>>    of their neighbour ETRs; and checking for outdated mappings and
>>    requesting updates.
> 
> Remove "neighbour". ETRs are anything but close to the ITRs.
> 
>>    In the ETR, control plane functions include participating in the
>>    neighbour reachability and liveness function (see Section 12.4);
>>    interacting with the mapping sub-system (next section); and answering
>>    requests for mappings (ditto).
>> 
>>    [[S3: Missing anything here?  Is this too brief, should we add a bit
>>    more meat?]]
> 
> Yes, missing a big part. Registering their mappings to the mapping system. That is what ETRs do in the control plane. And they also process Map-Requests and send Map-Replies.
> 
>>    first, [Iannone], was performed in the very early stages of the
>>    LISP effort, to verify that that approach was feasible.  First,
>>    packet traces of all traffic over the external connection of a large
>>    university (around 10,000 users) over a week-long period were
>>    collected.  Simulations driven by these recording were then
>>    performed; a variety of control settings on the cache were used, to
>>    study the effects of varying the settings.  The simulations set no
>>    limit on the total cache size, but used a range of cache retention
>>    times (i.e. an entry that remained unused longer than a fixed
>>    retention time was discarded), from 3 minutes, up to 300 minutes.
>> 
>>    First, the simulation gave the cache sizes that would result from
>>    such a cache design.  It showed that the resulting cache sizes ranged
>>    from 7,500 entries (at night, with the shortest retention time) up to
>>    about 100,000.  Using some estimations as to i) how many RLOCs the
>>    average mapping would have (since this will affect its size), and ii)
>>    how much memory it would take to store a mapping, this indicated
>>    cache sizes of between roughly 100 Kbytes and a few Mbytes.
> 
> This is too detail and the data may change in the future to obsolete the numbers provided above. Just summarize the conclusion of his research.
> 
>>    Of more interest, in a way, were the results regarding two important
>>    measurements of the effectiveness of the cache: i) the hit ratio
>>    (i.e. the share of references which could be satisified by the
>>    cache), and ii) the miss _rate_ (since control traffic overhead is
>>    one of the chief concerns when using a cache).  These results were
>>    also encouraging: miss (and hence lookup) rates ranged (again,
>>    depending on the time of day, cache settings, etc) from 30 per
>>    minute, up to 3,000 per minute (i.e. 150 per second; with the
>>    shortest timeout, and thus the smallest cache).  Significantly, this
>>    was substantially lower than the amount of observed DNS traffic,
>>    which ranged from 1,800 packets per minute up to 15,000 per minute.
> 
> Summarize here too.
> 
>>    The second, [Kim], was in general terms similar, except that it used
>>    data from a large ISP (taken over two days, at different times of the
>>    year), one with about three times as many users as the previous
>>    study.  It used the same cache design philosophy (the cache size was
>>    not fixed), but slightly different, lower, retention time values: 60
>>    seconds, 180 seconds, and 1,800 seconds (30 minutes), since the
>>    previous study had indicated that extremely long times (hours) had
>>    little additional benefit.
>> 
>>    The results were similar: cache sizes ranges from 20,000 entries with
>>    the shortest timeout, to roughly 60,000 with the longest; the miss
>>    rate ranged from very roughly 400 per minute (with the longest
>>    timeout) to very roughly 7,000 per minute (with the shortest),
>>    similar to the previous results.
> 
> There will be a lot more research so citing just these 4 will make this document incomplete. We don't want that. I think you should summarize what properties were considered and what high-level conclusions were made.
> 
>> 
>> 6.2.2.  Interface to the Mapping System
> 
> Earlier in this commentary, I suggested "API". If you consider it, this is where you want to put it. In this section.
> 
>> 
>>    The client interface to the indexing system from an ITR's point of
>>    view is not with the indexing sub-system directly; rather, it is
>>    through the client-interface sub-system, which is provied by devices
>>    called Map Resolvers (MRs).
>> 
>>    ITRs send request control messages (Map-Request packets) to an MR.
>>    (This interface is probably the most important standardized interface
>>    in LISP - it is the key to the entire system.)
>> 
>>    The MR then uses the indexing sub-system to allow it to forward the
>>    Map-Request to the appropriate ETR.  The ETR formulates reply control
>>    messages (Map-Reply packets), which are sent to the ITR.  The details
>>    of the indexing system are thus hidden from the ITRs.
>> 
>>    Similarly, the client interface to the indexing system from an ETR's
>>    point of view is through devices called Map Servers (MSs - admittedly
>>    a poorly chosen term, since their primary function is not to respond
>>    to queries, but it's too late to change it now).
> 
> I would strike the text in the parens. They are serving registrations so the term would be good as anything else.
> 
>>    ETRs send registration control messages (Map-Register packets) to an
>>    MS, which makes the information about the mappings which the ETR
>>    indicates it is authoritative for available to the indexing system.
>>    The MS formulates a reply control message (the Map-Notify packet),
> 
> Change "Map-Notify" to "Map-Reply".
> 
>> 6.2.3.1.  DDT Overview
>> 
>>    Conceptually, DDT is fairly simple: like DNS, in DDT the delegation
>>    of the EID namespace ([Perspective], Section "Namespaces-XEIDs") is
>>    instantiated as a tree of DDT 'nodes', starting with the 'root' DDT
>>    node.  Each node is responsible (authoritative?) for one or more
>>    blocks of the EID namespace.
>> 
>>    The 'root' node is reponsible for the entire namespace; any DDT node
>>    can 'delegate' part(s) of its block(s) of the namespace to child DDT
>>    node(s).  The child node(s) can in turn further delgate (necessarily
>>    smaller) blocks of namespace to their children, through as many
>>    levels as are needed (for operational, administrative, etc, needs).
>> 
>>    Just as with DNS, for reasons of performance, reliability and
> 
> Do you want to mention that DDT security is built into the design from day-1?
> 
>>    robustness, any particular node in the DDT delegation tree may be
>>    instantiated in more than one redundant physical server machines.
>>    Obviously, all the servers which instantiate a particular node in the
>>    tree have to have identical data about that node.
> 
> Don't use "server" here because it could imply or mislead the reader to think "Map-Servers". I think the best noun is "DDT-nodes" or describe from a parent node's reference and call them child nodes as you did in the previous paragraph.
> 
>>    Also, although the delegation hierarchy is a strict tree {{check - do
>>    all servers for the delegation of block X have to return the same
>>    list of servers for that block?}}, a single DDT server could be
>>    responsible (authoritative?) for more than one block of the EID
>>    namespace.
> 
> Yes, they do. But when they don't, when a Map-Request goes to one that doesn't have consistent config with the other sibling a delegation-hole or non-authoritative Map-Referral will be returned.
> 
>>    Eventually, leaf nodes in the DDT tree assign ({{delegate? - it's all
>>    static configured, nothing is dynamic}}) EID namespace blocks to
>>    MS's, which are DDT terminal nodes; i.e. a leaf of the tree is
>>    reached when the delegation points to an MS instead of to another DDT
>>    node.
> 
> Yes, all static configured.
> 
>>    The MS is in direct communication with the ETR(s) which both i) are
>>    authoritative for the mappings for that block, and ii) handle traffic
>>    to that block of EID namespace.
>> 
>> 6.2.3.2.  Use of DDT by MRs
>> 
>>    An MR which wants to find a mapping for a particular EID first
>>    interacts with the nodes of the DDT tree, discovering (by querying
>>    DDT nodes) the chain of delegations which cover that EID.  Eventually
>>    it is directed to an MS, and then to an ETR which is responsible
>>    {{authoritative?}} for that EID.
>> 
>>    Also, again like DNS, MRs cache information about the delegations in
>>    the DDT tree.  This means that once an MR has been in operation for
>>    while, it will usually have much of the delegation information cached
>>    locally (especially the top levels of the delegation tree).  This
>>    allows them, when passed a request for a mapping by an ITR, to
>>    usually forward the mapping request to the appropriate MS without
>>    having to do a complete tree-walk of the DDT tree to find any
>>    particular mappping.
>> 
>>    Thus, a typical resolution cycle would usually involve looking at
>>    some locally cached delegation information, perhaps loading some
>>    missing delegation entries into their delegation cache, and finally
>>    sending the Map-Request to the appropriate MS.
>> 
>>    [[S7: Forward ref to 'Scalabilty' for more on caching?  Should this
>>    discussion be here at all, or later?]]
> 
> What needs to be said from a high-level is that mappings can change at high frequency but the delegation tree does not change at *any* frequency since it is configured and organized based on EID allocation. This is why the mapping system scales. There is total mobility at the end-nodes (running LISP-MN or hosts behind xTRs) but the hierarchy does not need to reflect it.
> 
>>    The big advantage of DDT over the ALT, in performance terms, is that
>>    it allows MRs to interact _directly_ with distant DDT nodes (as
>>    opposed to the ALT, which _always_ required mediation through
>>    intermediate nodes); caching of information about those distant nodes
>>    allows DDT to make extremely effective use of this capability.
> 
> Right, the path of a Map-Request takes the same multi-hop path where in DDT is is optimized (to that picture above).
> 
>> 7.  Examples of Operation
>> 
>>    To aid in comprehension, a few examples are given of user packets
>>    traversing the LISP system.  The first shows the processing of a
>>    typical user packet, i.e. what the vast majority of user packets will
>>    see.  The second shows what happens when the first packet to a
>>    previously-unseen ultimate destination (at a particular ITR) is to be
>>    processed by LISP.
>> 
>>    [[E0: Should this be moved lover down, e.g. below "The Mapping
>>    System", once we've described the Map-Request, Map-Reply, etc control
>>    packets in a little more detail?]]
> 
> I think it should follow natural flow. So keep it the way it is.
> 
>>    [[E1: Also, these examples are all about operation in the _I_nternet;
>>    since many uses of LISP are in intranets, maybe we should have an
>>    example of that, too?]]
>> 
>> 7.1.  An Ordinary Packet's Processing
>> 
>>    This case follows the processing of a typical user packet (for
>>    instance, a normal TCP data or acknowledgment packet associated with
>>    an already-open TCP connection) as it makes its way from the original
>>    source host to the ultimate destination.
> 
> A TCP Syn or data packet has no special difference at the network layer. So why do you have to say this?
> 
>>    When the packet has made its way through the local site to an ITR
>>    (which is also a border router for the site), the border router looks
> 
> The ITR does not need to be a border router for the site. It could be directly connected to a source, or even a PE router. So just strike the text in the parens.
> 
>>    up the desination address (an EID) in its local mapping cache.  It
>>    finds a mapping, which instructs it to wrap the packet in an outer
>>    header (an IP packet, containing a UDP packet which contains a LISP
>>    header, and then the user's original packet).  The destination
>>    address in the outer header is set by the ITR to the RLOC of the
>>    destination ETR.
>> 
>>    The packet is then sent off through the Internet, using normal
>>    Internet routing tables, etc.
>> 
>>    On arrival at the destination ETR, the ETR will notice that it is
>>    listed as the destination in the outer header.  It will examine the
>>    packet, detect that it is a LISP packet, and unwrap it.  It will then
>>    examine the header of the user's original packet, and forward it
>>    internally, through the local site, to the ultimate destination.
>> 
>>    At the ultimate destination, the packet will be processed, and may
>>    produce a return packet, which follows the exact same process in
>>    reverse - with the exception that the roles of the ITR and ETR are
>>    swapped.
>> 
>>    [[E2: Should we have a more complicated example, e.g.with the return
>>    ITR&ETR not being the same as the forward path?]]
> 
> No, 6830 gives more detail. This is fine.
> 
>> 7.2.  A Mapping Cache Miss
>> 
>>    If a host sends a packet, and it gets to the ITR, and the ITR both i)
>>    determines that it needs to perform LISP processing on the user data
>>    packet, but ii) does not yet have a mapping cache entry which covers
>>    that destination EID, then more complex processing ensues.
> 
> Change "complex" to "additional".
> 
>>    It sends a Map-Request packet, giving the destination EID it needs a
>>    mapping for, to its MR.  The MR will look in its cache of delegation
>>    information to see if it has the RLOC for the ETR for that
> 
> The MR does not have a cache with ETR RLOCs, it has a cache with DDT-node and Map-Server addresses. Those addresses are in RLOC space.
> 
>>    destination EID.  If not, it will query the DDT system to find the
>>    RLOC of the ETR.  When it has the RLOC, it will send the Map-Request
>>    on to the ETR.
> 
> You should simply say that the MR forwards a Map-Request to the RLOC in its delegation cache. That could be a root, a intermediate DDT-node, or a Map-Server.
> 
>>    The ETR sends a Map-Reply to the ITR which needs the mapping; from
>>    then on, processing of user packets through that ITR to that ultimate
>>    destination proceeds as above.  (Typically, like many ARP
>>    implementations, the original user packet will have been discarded,
>>    not cached waiting for the mapping to be found.  When the host
>>    retransmits the packet, the mapping will be there, and the packet
>>    will be forwarded.)
> 
> I would strike the text in the parens. And add a short paragraph indicating the packet could be dropped, queued, or sent natively. Where the latter means it would flow toward a PITR.
> 
>>    [[E3: Should we given the detail on at least on DDT step?]]
> 
> I think you missed explaining that if a Map-Request is not going to a Map-Server that intermediate DDT-nodes return referrals. And when you explain that you should indicate that they provide keys for the children so subsequent Map-Referrals can be verified.
> 
>>    Use of UDP (instead of, say, a LISP-specific protocol number) was
>>    driven by the fact that many devices filter out 'unknown' protocols,
>>    so adopting a non-UDP encapsulation would have made the initial
>>    deployment of LISP harder - and our goal (see Section 3.1) was to
>>    make the deployment as easy as possible.
>> 
>>    The UDP source port in the encapsulated packet is a hash of the
>>    original source and ultimate destination; this is because many ISPs
> 
> It is a 5-tuple hash of the inner header.
> 
>>    use multiple parallel paths (so-called 'Equal Cost Multi-Path'), and
>>    load-share across them.  Using such a hash in the source-port in the
>>    outer header both allows LISP traffic to be load-shared, and also
>>    ensures that packets from individual connections are delivered in
>>    order (since most ISPs try to ensure that packets for a particular
>>    {source, source port, destination, destination port} tuple flow along
>>    a single path, and do not become disordered)..
>> 
>>    The UDP checksum is zero because the inner packet usually already has
>>    a end-end checksum, and the outer checksum adds no value.  [Saltzer]
>>    In most exising hardware, computing such a checksum (and checking it
>>    at the other end) would also present an intolerable load, for no
>>    benefit.
> 
> I am feeling this is too much detail. Why does someone who wants an intro need to know the detail about a checksum?
> 
>>    Note that lack of a response is not necessarily _proof_ that
>>    something has gone wrong - but it stronly suggests that something
>>    has, so other actions (e.g. a switch to an alternative ETR, if one is
>>    listed; a direct probe; etc) are advised.
> 
> Change to ".. a direct RLOC probe".
> 
>>    (See Section 12.5 for more about Echo Nonces.)
>> 
>> 9.3.3.  Instances
>> 
>>    Another use of these header fields is for 'Instances' - basically,
>>    support for VPN's across backbones.  [RFC4026] Since there is only
>>    one destination UDP port used for carriage of user data packets, and
>>    the source port is used for multiplexing (above), there is no other
>>    way to differentiate among different destination address namespaces
>>    (which are often overlapped in VPNs).
>> 
>> 9.4.  Probing
>> 
>>    RLOC-Probing (see [LISP], Section 6.3.2.  "RLOC-Probing Algorithm"
>>    for details) is a mechanism method that an ITR can use to determine
>>    with certainty that an ETR is up and reachable from the ITR.  As a
>>    side-benfit, it gives a rough RTT estimates.
>> 
>>    It is quite a simple mechanism - an ITR simply sends a specially
>>    marked Map-Request directly to the ETR it wishes information about;
>>    that ETR sends back a specially marked Map-Reply.  A Map-Request and
>>    Map-Reply are used, rather than a special probing control-message
>>    pair, because as a side-benefit the ITR can discover if the mapping
>>    has been updated since it cached it.
>> 
>>    The probing mechanism is rather heavy-weight and expensive (compared
> 
> You said it was simple but then you say heavy-weight. What you want to say is if the map-cache is large, a lot of messages will be required to be sent. To make it scale, you have to spread the sending of RLOC-probes over time. Which then affects the down detection convergence.
> 
> But indicate that Echo Noncing can help scale RLOC-probes.
> 
>>    to mechanisms like the Echo-Nonce), since it costs a control message
>>    from each side, so it should only be used sparingly.  However, it has
>>    the advantages of providing information quickly (a single RTT), and
>>    being a simple, direct robust way of doing so.
> 
> And note that Echo Noncing in the data-plane can be faster assuming bidirectional traffic.
> 
>> 9.5.  Mapping Lifetimes and Timeouts
>> 
>>    Mappings come with a Time-To-Live, which indicate how long the
>>    creator of the mapping expects them to be useful for.  The TTL may
>>    also indicate that the mapping should not be cached at all, or it can
>>    indicate that it has no particular lifetime, and the recipient can
>>    chose how long to store it.
>> 
>>    Mappings might also be discarded before the TTL expires, depending on
>>    what strategies the ITR is using to maintain its cache; if the
>>    maximum cache size is fixed, or the ITR needs to reclaim memory,
>>    mappings which have not been used 'recently' may be discarded.
>>    (After all, there is no harm in so doing; a future reference will
>>    merely cause that mapping to be reloaded.)
> 
> And the RLOC-set can change before the map-cache entry times out.
> 
>> 9.6.  Security of Mapping Lookups
>> 
>>    LISP provides an optional mechanism to secure the obtaining of
>>    mappings by an ITR.  [LISP-SEC] It provides protection against
>>    attackers generating spurious Map-Reply messages (including replaying
>>    old Map-Replies), and also against 'over-claiming' attacks (where a
>>    malicious ETR by claims EID-prefixes which are larger what what have
>>    been actually delegated to it).
>> 
>>    Very briefly, the ITR provided a One-Time Key with its query; this
> 
> Change "query" to "Map-Request".
> 
>>    key is used by both the MS (to verify the EID block that it has
>>    delegated to the ETR), and indirectly by the ETR (to verify the
>>    mapping that it is returning to the ITR).
> 
> The ETR uses the MS's OTK to sign the block. The ETR does not know the ITRs OTK and doesn't verify anything. The ITR verifies the signed Map-Reply by generating the MS OTK from its OTK (like the MS does) and uses the MS OTK to verify the signature, since that was the same key the ETR used.
> 
>>    In the other, referred to as the 'stateless' approach, for IPv4
>>    packets without the 'DF' bit set, too-large packets are fragmented,
>>    and then the fragments are forwarded; all other packets are
>>    discarded, and an ICMP Too Big message returned.
>> 
>>    [[X3: Spec is unclear about who reassembles; says "fragments are
>>    reassembled at the destination host" but also "reassembly can happen
>>    at the ETR" - I would have thought the latter was very undesirable,
>>    for all the obvious reasons, unless the dest cannot reassemble?]]
> 
> Right because if a core router fragments, it is fragmenting a packet destined to the ETR.
> 
>>    It is not clear at this point which approach is preferable.  [[X4:
>>    Also mention the Templin MTU scheme?  Find URL for message in which
>>    he discusses it, ref that way?  Or via the RANGER/whatever spec?]]
>> 
>> 10.  The Mapping System
>> 
>>    RFC 1034 ("DNS Concepts and Facilities") has this to say about the
>>    DNS name to IP address mapping system:
>> 
>>      "The sheer size of the database and frequency of updates suggest
>>      that it must be maintained in a distributed manner, with local
>>      caching to improve performance. Approaches that attempt to
>>      collect a consistent copy of the entire database will become more
>>      and more expensive and difficult, and hence should be avoided."
>> 
>>    and this observation applies equally to the LISP mapping system.
>> 
>>    To recap, the mapping system is split into an indexing sub-system,
>>    which keeps track of where all the mappings are kept, and the
>>    mappings themselves, the authoritative copies of which are always
>>    held by ETRs.
>> 
>>    [[M0: Should we mention, somewhere, the cases where they aren't -
>>    i.e. proxy map-replying?]]
> 
> I think you could have reference proxy map-replying in the DDT example. That way you can say that the MS either forwards the Map-Request to the MS or proxy reply itself.
> 
>> 10.1.2.1.  Solicit-Map-Request Messages
>> 
>>    "Solicit-Map-Request" (SMR) messages are actually not another message
>>    type, but a sub-type of Map-Reply messages.  They include a special
>  
> SMRs are Map-Requests because you want to use the long nonce as well. So an SMR is a Map-Request that is responded to by another Map-Request with the SMR-invoked bit set. And the nonce from the SMR is returned in the SMR-invoked Map-Request.
> 
>>    flag which indicates to the recipient that it _should_ send a new
>>    Map-Request message, to refresh its mapping, because the ETR has
>>    detected that the one it is using is out-dated.
>> 
>>    SMR's, like most other control traffic, is rate-limited. {{Need to
>>    say more about rate limiting, probably in security section?  Ref to
>>    that from here.}}
>> 
>> 10.1.3.  Map-Register and Map-Notify Messages
>> 
>>    The Map-Register message contains authentication information, and a
>>    number of mapping records, each with an individual Time-To-Live
>>    (TTL).  Each of the records contains an EID (potentially, a block of
>>    EIDs) and its AFI, a version number for this mapping (see
>>    Section 9.3.1), and a number of RLOCs and their AFIs.
>> 
>>    Each RLOC entry also includes the same data as in the Map-Replies
>>    (i.e. priority and weight); this is because in some circumstances it
>>    is advantageous to allow the MS to proxy reply on the ETR's behalf to
>>    Map-Request messages.  [Mobility]
> 
> Also note an RLOC can be a multi-tuple encoding meaning it can return, for example, a Geo-Tag, or ELP or a RLE (replication list entry for multicast).
> 
>>    Map-Notify messages have the exact same contents as Map-Register
>>    messages; they are purely acknowledgements.
> 
> They are not just acknowledgements. They are used to tell the old RLOC-set that new RLOCs have been registered.
> 
>> 10.2.  The DDT Indexing Sub-system
>> 
>>    As previously mentioned Section 6.2.3, the indexing sub-system in
>>    LISP is currently the DDT system.
>> 
>>    The overall operation is fairly simple; an MR which needs a mapping
>>    starts at a server for the root DDT node (there will normally be more
>>    than one such server available, for both performance and robustness
>>    reasons), and through a combination of cached delegation information,
>>    and repetitive querying of a sequence of DDT servers, works its way
>>    down the delegation tree until it arrives at an MS which is
>>    authoritative (responsible?) for the block of EID namespace which
>>    holds the destination EID in question.
>> 
>>    The interaction between MRs and DDT servers is not complex; the MR
>>    sends the DDT server a Map-Request control message (which looks
>>    almost exactly like the Map-Request which an ITR sends to an MR).
> 
> Don't need to say the text in the parens. And again use "DDT node" and not "DDT server".
> 
>>    The DDT server uses its data (which is configured, and static) to see
>>    whether it is directly peered to an MS which can answer the request,
>>    or if it has a child (or children, if replicated) which is
>>    responsible for that portion of the EID namespace.
>> 
>>    If it has children which are responsible, it will reply to the MR
>>    with another kind of LISP control message, a Map-Referral message,
> 
> Change to "... has children node configured, ...".
> 
>>    which provides information about the delegation of the block
>>    containing the requested EID.  The Map-Referral also gives the RLOCs
>>    of all the machines which are DDT servers for that block. and the MR
>>    can then send Map-Requests to any one (or all) of them.
>> 
>>    Control flags in the Map-Referral indicate to the querying MR whether
>>    the referral is to another DDT node, an MS, or an ETR.  If the
>>    former, the MR then sends the Map-Request to the child DDT node,
>>    repeating the process.
>> 
>>    If the latter, the MR then interacts with that MS, and usually the
>>    block's ETR(s) as well, to cause a mapping to be sent to the ITR
>>    which queried the MR for it.  (Recall that some MS's provide Map-
>>    Replies on behalf of an associated ETR, so in such cases the Map-
>>    Reply will come from the MS, not the ETR. {{I think this case has
>>    been mentioned already; check.}})
> 
> This is the place to put the public key description in and how the MR will verify a Map-Referral.
> 
>>    Delegations are cached in the MRs, so that once an MR has received
>>    information about a delegation, it will not need to look that up
>>    again.  Once it has been in operation for a short while, it will only
>>    need to ask for delegation information which is has not yet asked
>>    about - probably only the last stage in a delegation to a 'leaf' MS.
>> 
>>    As describe below (Section 10.6), significant amounts of modeling and
>>    performance measurement have been performed, to verify that DDT has
>>    (and will continue to have) acceptable performance.
>> 
>> 10.2.1.  Map-Referral Messages
>> 
>>    Map-Referral messages look almost identical to Map-Reply messages
>>    (which is felt to be an advantage by some people, although having a
>>    more generic record-based format would probably be better in the long
>>    run, as ample experience with DNS has shown), except that the RLOCs
>>    potentially name either i) other DDT nodes (children in the
>>    delegation tree), or ii) terminal MSs.
> 
> There is no need to have this section I think. And don't criticize the architecture you are describing.  ;-)
> 
>> 10.3.  Reliability via Replication
>> 
>>    Everywhere throughout the mapping system, robustness to operational
>>    failures is obtained by replicating data in multiple instances of any
>>    particular node (of whatever type).  Map-Resolvers, Map-Servers, DDT
>>    nodes, ETRs - all of them can be replicated, and the protocol
>>    supports this replication.
>> 
>>    The deployed DDT system actually uses anycast [RFC4786], along with
>>    replicated servers, to improve both performance and robustness.
>> 
>>    There are generally no mechanisms specified yet to ensure coherence
>>    between multiple copies of any particular data item, etc - this is
>>    currently a manual responsibility.  If and when LISP protocol
>>    adoption proceeds, an automated layer to perform this functionality
>>    can 'easily' be layered on top of the existing mechanisms.
> 
> I do not understand what you mean here. And therefore, why it is even necessary to have multiple copies.
> 
>> 
>> 10.4.  Security of the DDT Indexing Sub-system
>> 
>>    LISP provides an advanced model for securing the mapping indexing
>>    system, in line with the overall LISP security philosophy.
>> 
>>    Briefly, securing the mapping indexing system is broken into two
>>    parts: the interface between the clients of the system (MR's) and the
>>    mapping indexing system itself, and the interaction between the DDT
>>    nodes/servers which make it up.
>> 
>>    The client interface provides only a single model, using the
>>    'canonical' public-private key system (starting from a trust anchor),
>>    in which the child's public key is provided by the parent, along with
>>    the delegation.  This requires very little configuration in the
>>    clients, and is fairly secure.
>> 
>>    The interface between the DDT nodes/servers allows for choices
>>    between a number of different options, allowing the operators to
>>    trade off among configuration complexity, security level, etc.  This
>>    is based on experience with DNS-SEC ([RFC4033]), where configuration
>>    complexity in the servers has been a major stumbling block to
>>    deployment.
> 
> There isn't any client interface. There is just the Map-Referrals that the DDT-nodes use. So I am not sure what you mean but the text is confusing. Just indicate that a parent will provide the public key of the children so when the children sign a Map-Referral, the MR can verify it.
> 
>>    See [Perspective], Section "Security-Mappings" for more.
>> 
>> 10.5.  Extended Tools
>> 
>>    In addition to the priority and weight data items in mappings, LISP
>>    offers other tools to enhance functionality, particularly in the
>>    traffic engineering area.
>> 
>>    One is 'source-specific mappings', i.e. the ETR may return different
>>    mappings to the enquiring ITR, depending on the identity of the ITR.
>>    This allows very fine-tuned traffic engineering, far more powerful
>>    than routing-based TE.
>> 
>>    [[M3: Examples needed?  How about something about why one would want
>>    to do this?  Should I talk about how this will negatively impact the
>>    ability to cache mappings in MRs, unless we add 'applicability
>>    ranges' to Map-Replies?]]
>> 
>>    [[M4: Any more?]]
> 
> There are examples in the TE draft.
> 
>> 10.6.  Performance of the Mapping System
>> 
>>    Prior to the creation of DDT, a large study of the performance of the
>>    previous mapping system, ALT ([ALT]), along with a proposed new
>>    design called TREE (which used DNS to hold delegation information)
>>    provided considerable insight into the likely performance of the
>>    mapping systems at larger scale.  [Jakab] The basic structure and
>>    concepts of DDT are identical to those of TREE, so the performance
>>    simulation work done for that design applies aequally to DDT.
>> 
>>    In that study, as with earlier LISP performance analyses, extensive
>>    large-scale simulations were driven by lengthy recordings of actual
>>    traffic at several major sites; one was the site in the first study
>>    ([Iannone]), and the other was an even large university, with roughly
>>    35,000 users.
>> 
>>    The results showed that a system like DDT, which caches information
>>    about delegations, and allows the MR to communicate directly with the
>>    lower nodes on the delegation hierarchy based on cached delegation
>>    information, would have good performance, with average resolution
>>    times on the order of the MR to MS RTT.  This verified the
>>    effectiveness of this particular type of indexing system.
>> 
>>    A more recent study, [Saucez], has measured actual resolution times
>>    in the deployed LISP network; it took measurements from a variety of
>>    locations in the Internet, with respect to a number of different
>>    target EIDs.  Average measured resolution delays ranged from roughly
>>    175 msec to 225 msec, depending on the location.
>> 
>>    [[M5: Perhaps this belongs in "Scalability"?]]
> 
> Yes, put all scalability in one place.
> 
>>    [[M6: What about potential caching in MRs?]]
> 
> Don't document it because you will set an expectation and readers won't find it in any of the RFCs.
> 
>> 11.  Deployment Issues and Mechanisms
>> 
>>    This section discusses several deployment issues in more detail.
>>    With LISP's heavy emphasis on practicality, much work has gone into
>>    making sure it works well in the real-world environments most people
>>    have to deal with.
>> 
>> 11.1.  LISP Deployment Needs
>> 
>>    As mentioned earlier (Section 3.2), LISP requires no change to almost
>>    all existing hosts and routers.  Obviously, however, one must deploy
>>    _something_ to run LISP!  Exactly what that has to be will depend
>>    greatly on the details of the site's existing networking gear, and
>>    choices it makes for how to achieve LISP deployment.
>> 
>>    The primary requirement is for one or more xTRs.  These may be
>>    existing routers, just with new software loads, or it may require the
>>    deployment of new devices.
>> 
>>    LISP also requires a small amount of LISP-specific support
> 
> Don't say small, that might not be true. Just say new devices are added (versus existing devices have to change).
> 
>>    If an ITR is holding a seriously outdated cached mapping, it may send
>>    packets to an ETR which is no longer an ETR for that EID.
> 
> Remove "seriously". The TTL has either expired or not.
> 
>>    It might be argued that if the ETR is properly managing the lifetimes
>>    on its mapping entries, this 'cannot happen', but it is a wise design
>>    methodology to assume that 'cannot happen' events will in fact happen
>>    (as they do, due to software errors, or, on rare occasions, hardware
>>    faults), and ensure that the system will handle them properly (if,
>>    perhaps not in the most expeditious, or 'clean' way - they are, after
>>    all, very unlikely to happen).
>> 
>>    ETRs can easily detect cases where this happpens, after they have un-
>>    wrapped a user data packet; in response, they send a Solicit-Map-
>>    Request to the source ITR to cause it to refresh its mapping.
>> 
>>    [[F4: The LISP spec is not clear (at least, on a quick reading) on
>>    whether ETRs should check for this, and what their response should
>>    be.  According to Dino, in most cases ETRs can detect that they are
>>    the wrong ETR, but I need to finalize that discussion.]]
> 
> They need state from the mapping system that tells them an EID-prefix has moved. That is what tells an ETR is is the wrong one.
> 
>> 12.2.3.  Outdated Mappings - No Longer an ETR
>> 
>>    In another case for what can happen if an ITR uses an outdated
>>    mapping, the destination of traffic from an ITR might no longer be a
>>    LISP device at all.  In such cases, one might get an ICMP Destination
>>    Unreachable error message.  However, one cannot depend on that - and
>>    in any event, that would provide an attack vector, so it should be
>>    used with care.  (See [LISP], Section 6.3, "Routing Locator
>>    Reachability" for more about this.)
>> 
>>    The following mechanism will work, though.  Since the destination is
>>    not an ETR, the echoing reachability detection mechanism (see
>>    Section 9.3.2) will detect a problem.  At that point, the backstop
>>    mechanism, Probing, will kick in.  Since the destination is still not
>>    an ETR, that will fail, too.
>> 
>>    At that point, traffic will be switched to a different ETR, or, if
>>    none are available, a reload of the mapping may be initiated.  [[F7:
>>    Does that actually happen?]]
>> 
>> 12.3.  Erroneous Mappings
>> 
>>    Again, this 'should not happen', but a good system should deal with
>>    it.  However, in practise, should this happen, it will produce one of
>>    the prior two cases (the wrong ETR, or something that is not an ETR),
>>    and will be handled as described there.
>> 
>>    [[F8: I suppose if one ETR is handing out bad mappings, it might be
>>    nice to be able to bypass it.  This probably falls under 'Future
>>    Work', though.]]
> 
> This section sounds like a requirement and not a description of what can happen. I think it should be removed.
> 
>> 12.4.  Neighbour Liveness
> 
> RLOC liveness.
> 
>>    The ITR, like all packet switches, needs to detect, and react, when
>>    its next-hop neighbour ceases operation.  As LISP traffic is
> 
> It is not a next-hop. Have to use "RLOC" or "remote RLOC".
> 
>>    effectively always unidirectional (from ITR to ETR), this could be
>>    somewhat problematic.
>> 
>>    Solving a related problem, neighbour reachability (below) subsumes
>>    handling this fault mode, however.
>> 
>>    Note that the two terms (liveness and reachability) are _not_
>>    synonmous (although a lot of LISP documentation confuses them).
> 
> Remove the text in the parens.
> 
>> 12.5.  Neighbour Reachability
>> 
>>    [[F9: Maybe some of this belongs in "xTRs-Channel-Echo"?  I'm also
>>    wondering if it belongs in the architecture document, since it's
>>    fairly general, but.... it quickly gets down to mechanism details?]]
>> 
>>    A more significant issue than whether a particular ETR E is up or not
>>    is, as mentioned above, that although ETR E may be up, attached to
>>    the network, etc, an issue in the network between a source ITR I and
>>    E may prevent traffic from I from getting to E. (Perhaps a routing
>>    problem, or perhaps some sort of access control setting.)
>> 
>>    The one-way nature of LISP traffic makes this situation hard to
>>    detect in a way which is economic, robust and fast.  Two out of the
>>    three are usually not to hard, but all three at the same time - as is
>>    highly desirable for this particular issue - are harder.
>> 
>>    In line with the LISP design philosophy ([Perspective], Section
>>    "Design-Theoretical"), this problem is attacked not with a single
>>    mechanism (which would have a hard time meeting all those three goals
>>    simultaneously), but with a collection of simpler, cheaper
>>    mechanisms, which collectively will usually meet all three.
>> 
>>    They are reliance on the underlying routing system (which can of
>>    course only reliably provide a negative reachabilty indication, not a
>>    positive one), the echo nonce (which depends on some return traffic
>>    from the destination xTR back to the source xTR), and finally direct
>>    'pinging', in the case where no positive echo is returned.
>> 
>>    (The last is not the first choice, as due to the large fan-out
>>    expected of LISP devices, reliance on it as a sole mechanism would
>>    produce a fair amount of overhead.)
> 
> I think this section is redundant with the previous. If you think there are salient points in it, then move them to the prior parapgraph.
> 
>> 
>> 13.  On-Going Improvements
>> 
>>    In line with the philosophies laid out in [Perspective], Section
>>    "Design", LISP is something of a moving target.  This section
>>    discusses some of the contemporaneous improvements being made to
>>    LISP.
>> 
>> 13.1.  Improved NAT Support
>> 
>>    As indicated above (Section 11.3), initial support for operation of
>>    LISP in the presence of NAT devices was limited.  A more
>>    comprehensive approach to support of LISP xTR deployment behind NAT
>>    devices, involving a fairly extensive supplement to LISP, LISP NAT
>>    Traversal, has been designed.  [NAT-Traversal]
>> 
>>    A new class of LISP device, the LISP Re-encapsulating Tunnel Router
>>    (RTR), passes traffic through the NAT, both to and from the xTR.
>>    (Inbound traffic has to go through the RTR as well, since otherwise
>>    multiple xTRs could not operate behind a single NAT, for the
>>    'specified port' reason described in Appendix B.4.)
>> 
>>    (Had the Map-Reply included a port number, this could have been
>>    avoided - although of course it would be possible to define a new
>>    RLOC type which included protocol and port, to allow other
>>    encapsulation techniques.)  [[D6: Make that observation in the
>>    general design section, too.]]
>> 
>>    Two new LISP control messages (Info-Request and Info-Reply) allow an
>>    xTR to detect if it is behind a NAT device, and also discover the
>>    global IP address and UDP port assigned by the NAT to the xTR.  A
>>    modification to LISP Map-Register control messages allows the xTR to
>>    initialize mapping state in the NAT, in order to use the RTR.
>> 
>>    This mechanism addresses cases where the xTR is behind a NAT, but the
>>    xTR's associated MS is on the public side of the NAT; this
>>    limitation, that MS's must be in the 'public' part of the Internet,
>>    seems reasonable.
> 
> Don't say this is improved. It sounds like we had multiple stages. We actually did but no real customer implemented the first one so just say we have a NAT-traversal design and put the above description earlier in the paper.
> 
>> 
>> 13.2.  Mobile Device Support
>> 
>>    Mobility is an obvious capability to provide with LISP.  Doing so is
>>    relatively simple, if the mobile host is prepared to act as its own
>>    ETR.  It obtains a local 'temporary use' address, and registers that
>>    address as its RLOC.  Packets to the mobile host are sent to its
>>    temporary address, wherever that may be, and the mobile host first
>>    unwraps them (acting as an ETR), and the processes them normally
>>    (acting as a host).
>> 
>>    (Doing mobility without having the mobile host act as its ETR is
>>    difficult, even if ETRs are quite common.  The reason is that if the
> 
> That is not true. Cisco ships VMs moving where the VMs are assigned EIDs and the end-of-row or top-of-rack routers are the RLOCs.
> 
>>    ETR and mobile host are not integrated, during the step from the ETR
>>    to the mobile host, the packets must contain the mobile host's EID,
>>    and this may not be workable.  If there is a local router between the
>>    ETR and mobile host, for instance, it is unlikely to know how to get
>>    the packets to the mobile host.)
>> 
>>    If the mobile host migrates to a site which is itself a LISP site,
>>    things get a little more complicated.  The 'temporary address' it
>>    gets is itself an EID, requiring mapping, and wrapping for transit
>>    across the rest of the Internet.  A 'double encapsulation' is thus
>>    required at the other end; the packets are first encapsulated with
>>    the mobile node's temporary address as their RLOC, and then this has
>>    to be looked up in a second lookup cycle (see Section 9.1), and then
>>    wrapped again, with the site's RLOC as their destination.
> 
> Too detailed Noel. Just indicate that a LISP-MN will use an RTR to interact with through a NAT device. And to allow it to scale so when it moves, not all the xTRs for the EIDs the LISP-MN is talking to have to be updated, only the RTR does. So a big scaling benefit.
> 
>>    This results in slight loss in maximum packet size, due to the
>>    duplicated headers, but on the whole it is considerably simpler than
>>    the alternative, which would be to re-wrap the packet at the site's
>>    ETR, when it is discovered that the ultimate destination's EID was
>>    not 'native' to the site.  This would require that the mobile node's
>>    EID effectively have two different mappings, depending on whether the
>>    lookup was being performed outside the LISP site, or inside.
>> 
>>    {{Also probably need to mention briefly how the other end is notified
>>    when mappings are updated, and about proxy-Map-Replies.}} [Mobility]
> 
> Should remove as well.
> 
>> 13.3.  Multicast Support
>> 
>>    Multicast may seem an odd thing to support with LISP, since LISP is
>>    all about separating identity from location, but although a multicast
>>    group in some sense has an identity, it certainly does not have _a_
>>    location.
> 
> Try to rephrase. Multicast is important because EIDs want to send and receive multicast packets. And we have to deal with it.
> 
>>    However, multicast is important to some users of the network, for a
>>    number of reasons: doing multiple unicast streams is inefficient; it
>>    is easy to use up all the upstream bandwidth, and without multicast a
>>    server can also be saturated fairly easily in doing the unicast
>>    replication.  So it is important for LISP to 'play nicely' with
>>    multicast; work on multicast support in LISP is fairly advanced,
>>    although not far-ranging.
> 
> It doesn't play nicely with it. It has to support it.
> 
>>    Briefly, destination group addresses are not mapped; only the source
>>    address (when the original source is inside a LISP site) needs to be
>>    mapped, both during distribution tree setup, as well as actual
>>    traffic delivery.  In other words, LISP's mapping capability is used:
>>    it is just applied to the source, not the destination (as with most
>>    LISP activity); the inner source is the EID, and the outer source is
>>    the EID's RLOC.
> 
> Well be careful here because we have proposals to put (S,G) state in the mapping database that is "joined/registered" by ETRs so ITRs can find such entries to replicate and encapsulate to.
> 
>>    Note that this does mean that if the group is using separate source-
>>    specific trees for distribution, there isn't a separate distribution
>>    tree outside the LISP site for each different source of traffic to
>>    the group from inside the LISP site; they are all lumped together
>>    under a single source, the RLOC.
>> 
>>    The approach currently used by LISP requires no packet format changes
>>    to existing multicast protocols.  See [Multicast] for more;
>>    additional LISP multicast issues are discussed in [LISP], Section 12.
> 
> Don't say that here because if you use some of the other proposals and don't do PIM signaling from ETR to ITR then there are no changes require of PIM because PIM is not used.
> 
>> Appendix A.  Glossary/Definition of Terms
>> 
>>    [[g1: I _hate_ it when people put this up front, it really cuts into
>>    the flow]]
>>    [[g2: Also, most of these can probably be lifted directly from the
>>    LISP RFC.]]
> 
> Yes, I think you should.
> 
>>   LISP transitioned from an IRTF activity to an IETF WG in March 2009,
>>    and after numerous revisions, the basic specifications moved to
>>    becoming RFCs in 2012 (although work to expand and improve it, and
>>    find new uses for it, continues, and undoubtly will for a long time
>>    to come).
> 
> RFCs actually happened in Jan 2013.
> 
> I think the document is really in good shape.
> 
> Dino
>