Re: [Ila] [5gangip] Scaling mapping systems (was Re: BOF Description)

[Tom Herbert <[e-mail]>]

>
>
> In ILA there is likely a one-to-one association between locator and ID. So
> the out of order situation can happen to each new specific flow. In LISP,
> the locator can be a prefix used for many EIDs so the out of order or drop
> situation on the ITR only happens for one destination and not to all the
> others that start a flow for the same prefix.
>

That would also be true of ILA, flows going to the same destination will
share the same cache entry.

At the start of a flow (or resumption from idle) a connection should be in
slow start so that there wouldn't be a lot of packets inflight to create
OOO packets, or it might even be only a single SYN in flight. It's likely
that the RTT to get the redirect is less than the application response time
so that the cache would be populated before a sender sends its second round
of packets.

As you pointed out the alternative to redirect is do a query and either
drop packets or queue packets until there is a response from the mapping
server. Dropping is problematic because it can introduce latency, consider
that if an initial SYN is dropped the sender takes an initRTO which is at
least 1 sec delay. Queuing is a problem of resources management and the
queue becomes a possible DOS target.

>
> > > > trianglular routing. Redirects must be secure so that they cannot be
> spoofed, so for that reason (and some others) the protocol is over TCP. The
> mapping cache is only an optimization and packets are never dropped or
> queued for pending cache resolution. If the cache weren't present,
> communications would
> > >
> > > But the ILA router in the network needs to have the mapping. And if it
> does not, what happens then?
> > >
> > > The ILA router is considered authoritative for its shard. If it
> doesn't have a mapping then the packet is silently dropped as having an
> unreachable destination.
> >
> > Does the ILA router for the shard only route to destinations inside of
> the shard?
> >
> > Yes.
> >
> > And if so, it is kept small so you can push the mappings and make it
> scale at that level?
> >
> > Small is relative. The ILA router first needs to being able to forward
> the load, so provisioning would need to consider the worse case where no
> entries hit the cache. Redirects would likely be rate limited (probably
> capped by throughput). In a network, design I would probably target a cache
> hit rate of at least 95%.
>
> In any loc/ID design push can be used and scale. But is it practical (and
> necessary) for 1B entries?
>
> This isn't doing push in the pub/sub sense which I agree won't scale to 1B
entries. Redirect are reactions to forwarded packets so the packets
effectively are the mapping request.


> > > > still be viable but sub-optimal; that characteristic establishes a
> bound for the worst case DOS attack.
> > > >
> > > > Any thoughs on this approach?
> > >
> > > You’ll really need to put signatures in the Redirect messages if you
> want robust authentication.
> > >
> > > The redirects are sent over an established TCP connection to an ILA-N
> so that deters message spoofing. For stronger security, TCP authentication
> option or TLS can be used.
> >
> > Is it worth the state to hold millions of TCP connections only the
> occasion redirect message?
> >
> > It might be millions of TCP connections spread across a large network,
> but for ILA-Ns it's at most a couple of hundred connections and for a
> mapping resolution ILA-R it's probably no more than 50K. Assuming 2K memory
> per TCP connection ctx (and TLS if in use) that's a 100M on the ILA-Rs. In
> addition to security, the other reasons to use TCP is it's reliable
> transport and provides congestion control and avoidance. There's also the
> option of using TFO if the amount of state were to become an issue.
>
> Okay. Agree its not a memory-issue proble I was referring to. It was how
> to manage the connections. And how much is exposed to a network admin
> engineer to debug. Managing 100s of BGP connections have been a challenged
> with the customers I have worked with in the past.
>
> My sense is that network control protocols are trending to use TCP with an
RPC layer. For instance in the datacenter use case, an operator like Google
or Facebook are going to be most comfortable with a protocol over TCP using
a their common RPC and authentication infrastructure (i.e. Google would
probably want to use gRPC, Facebook probably Thrift as they seem to be
using in Open/R). For 3GPP 5G the service-based interfaces are HTTP 2.0
(REST) over TCP.

Tom