Re: [sidr] some comments and questions regarding rpki-rtr

[Rob Austein <[e-mail]>]

At Sat, 1 Oct 2011 11:42:24 +0200 (CEST), Tim Bruijnzeels wrote:
> 
> A = No changes
> =============
> So if I read 5.2 correctly the cache should respond with just 1
> end-of-data pdu if there are no updates since the serial included in a
> serial request.
> But if read 5.4 I can also interpret this that we should respond with 2
> pdus: 1 cache response, 0 data records, 1 end-of-data
> 
> Can you please tell me which is correct?

The intent is to send an empty update, ie, a normal update that
happens to have no data records because nothing has changed.

So it should be cache response followed by end of data.

> Like this?
> 
>    Cache                         Router
>      ~                             ~
>      | <----- Serial Query ------- | R requests data
>      |                             |
>      | ----- Cache Response -----> | C confirms request
>      | ------  End of Data ------> | C sends End of Data
>      |                             |   and sends *same* serial
>      ~                             ~

Yes.  This is what my implementation does.

> B = Nonce and cache reset
> =====================
> 
> When I read 5.10 the nonce is generated when the cache starts. And reading
> 6.3 the cache may send a cache reset reply to the client when there are no
> incremental updates available.
> 
> Does this imply that a new cache nonce should be generated?

No.  Server should generate a new nonce if and only if the serial
numbers it has been using up until now have become meaningless
(technical term used in the draft is "commensurate" -- nonce changes
when something happens that prevents serial numbers before and after
the event from being "commensurate", ie, measuring the same thing).

The cache nonce is really a light-weight session identifier.
Generating a new nonce is equivalent to terminating any existing
sessions that were using the old nonce.   Serial numbers are not
meaningful across sessions, so serial query across a nonce change is
not possible, router must do a cache reset.

> I assumed that it did not. The nonce is made when the process starts. So
> when a client sends a reset query the same nonce may be kept. The client
> just gets a new, full, data set, up to the current serial for that same
> nonce.

The server doesn't even have to generate a new nonce when it starts,
so long as the serial numbers are still commensurate.

My server generates a new nonce when it starts up if and only if it
has no record of what nonce and serial numbers it was using when it
was last running.

> Part of the reason I am asking is that we are currently not yet able to
> send incremental updates. So our cache always replies as described in 6.3.
> We are not resetting the nonce though, and we are seeing duplicate
> announcement errors from the routers.

You should not need to reset the nonce unless something has whacked
the serial numbers.

At least one of the router images currently under test is known to
generate spurious complaints about duplicate announcements, that may
be what you're seeing.  Check with whoever got you that image.

> C = Duplicate announcements / withdrawals
> ===================================
> 
> As described in 5.5:
>   The cache server MUST ensure that it has told the router client to
>    have one and only one IPvX PDU for a unique {prefix, len, max-len,
>    asn} at any one point in time.
> 
> So this means that cache should exclude duplicates in a full update even
> if the same unique {prefix, len, max-len, asn} exists more than once (same
> ROA, multiple prefixes, or different ROAs).
> 
> I probably missed the discussion on this, but can you explain why this is?
> I don't see a conflict. If I get the same announce twice, it's still just
> announce?

I don't entirely get the urgency behind this one myself, but my
understanding is that Randy and the router guys were concerned that
duplicates meant that the router and cache disagreed on current state,
therefore duplicates were an indication that something is wrong.

> I am also wondering what this means wrt serial updates. Let me clarify by
> example: 10/16 is announced in serial 2, withdrawn in 3, announced again
> in 4. The router has serial 1. Should the cache then work out the exact
> delta between 1 and 4, or can it send 1-2, followed by 2-3, followed by
> 3-4.

I work out the exact delta.  It's not that hard, just a bit tedious,
and I pre-compute it once when new updates come in from my validation
code rather than doing it on the fly in the server.

> I am afraid though, that this may cause scaling issues when a potentially
> large number of routers use the same cache (cpu), or a large number of
> pre-computed deltas need to be kept (memory). I think that if this
> responsibility were just handled by the routers we would have much better
> scaling on the cache side, and it would be much easier for caches to keep
> incremental updates without having to resort to no-incremental-updates
> like 6.3 describes.

Well, it's not necessarily memory.  I keep the pre-computed deltas on
disk.  How much data that works out to be depends on how far back one
is willing to keep the deltas.

The general assumption, though, is that servers are cheaper than
routers.  Server for this protocol can be a 600 USD 1U BSD or Linux
box; my understanding is that you're not likely to find serious
routers in that price range anytime soon. :)

Also, routers are supposed to spend their time moving packets, not
performing unnecessarily complex internal database manipulations.

So anything we can do on the server to make the router code simpler is
probably a win for the operator who has to pay for all this kit.

> D = Keep alive timeout
> ===============
> 
> As described in 6.1:
>    To limit the length of time a cache must keep the data necessary to
>    generate incremental updates, a router MUST send either a Serial
>    Query or a Reset Query no less frequently than once an hour.  This
>    also acts as a keep alive at the application layer.
> 
> So, we have interpreted this to say that it's probably good on our side to
> drop the connection after 1 hour. It's must likely dead and we want our
> resource back...
> 
> When we do this, do you think it would be good if we tried to send an
> error pdu just before closure? With a new error code indicating session
> timeout?

Um, the error PDU would almost certainly not get through, pretty much
by definition, because if all is well we would be seeing serial or
reset queries before the keep alive timer expires.

> E = Cache shutdown
> ==============
> 
> When the cache is stopped for whatever reason. Server restart, cache had
> irrecoverable internal error, anything else...
> 
> Should we send a new type of notify / error (with  new specific code) to
> all children so that they can gracefully switch over to another cache --
> or wait until we are back?
> 
> Or should we just close the connections?

I think just closing connections is sufficient.  Router has to cope
with connection just closing, what would it do differently here?