Re: [netmod] Common etag, timestamp on all interfaces (draft-lindblad-netconf-transaction-id)

[Jan Lindblad <[e-mail]>]

Andy, Balazs, Kent,

Thanks for your very good questions. Comments inline.
> I assume that the etag defined in your I-D is the same as the one defined in Restconf. Does or should your draft include a statement like:
> 
> “The etag values maintained by the server are protocol/interface independent. If requested the same etag values will be visible on all interface including Restconf, Netconf, CLI etc.”
> 
>  
> 
> While it makes sense that a server would use the same values across protocols, I'm unsure if it's needed and, if we do, if we could state it in a NETCONF-specific draft.
> 
> BALAZS2: I see it as a VERY important advantage of the whole YANG/Netconf/Restconf ecosystem that the separate protocols (practically including the CLI and possibly a gui too) are just views of the same central configuration datastore. So IMO this is important and should be stated.
> 
As an implementor, I think it makes great sense to use the same underlaying mechanism for all mgmt interfaces.

I could imagine some implementors planning to lean on the wire representation to compute hashes over the data as their ETag implementation. That would not be possible under this requirement, but ruling out that particular idea would not be problematic, imo.

Long ago, I pondered this question for a while, but then I wasn't able to think of a use case where a client would benefit from a guarantee that all interfaces have common ETags, so I left it out in order to not produce a CLR. I'm not against adding this requirement if we can think of a reasonable reason why. Adding a requirement for CLI+GUI sounds hard in practice, though.
> I strongly support this approach.
> It applies to the entire server API, including notifications.
> E.g., a client should be able to reuse code for processing NETCONF notifications,
> even if the protocol is RESTCONF or the new YANG-Push over UDP.
> 
> The RESTCONF mechanisms adapted from HTTP should be extended to be
> protocol-independent.  Our goal should be code to the YANG models, NOT the protocols.

Yes, I certainly like that. 
> Restconf also includes timestamps. What was your reason to exclude them from your I-D ? IMHO if the server maintains timestamps they would be protocol/interface independent just as etags, so the task is to make them available on Netconf too (and maybe the CLI).
> 
> I agree and have mentioned before.  LastModified either needs to be added, or justified why not added, to get my adoption support.
> 
I'm not against having Last-Modified in there. In fact, it was in the draft initially. Here's the justification why I removed it :-)

+ When I started the cost/benefit analysis for what I proposed, I soon found that it's not unimportant how many and how large ETag attributes are added to the payload. Having two mechanisms (i.e. both ETag and Last-Modified) doing pretty much the same thing started to look expensive in performance.
+ Since it's essential that the time stamp would have to be the same on all touched elements, all the components/subsystems/subagents involved would still have to be fed the exact time from the central management agent.
+ Then there's one or two more things with If-Unmodified-Since that I discuss below.

> I agree. They both need to be supported in RESTCONF.

RESTCONF has a SHOULD for the (single) datastore root level and MAY for the deeper levels. 

> A timestamp can be applied to multiple servers, unlike the ETag values.

I don't think this analysis is correct. A client connecting with 15 servers in a network wide transaction would surely receive several differing timestamps. The times may be fairly close if NTP is running fine, latency is low, network and cpu load is even and the servers are running similar code, but they will rarely all be the same even with the low 1s Last-Modified resolution. If you allow clients to set the ETag in the request (which is trivial to allow), then the tags could be exactly the same across all servers (and known to the client before the request is even sent to the servers, allowing request pipelining).

> Typical usage is for the client to track its own polling timestamps,
> and use If-Modified-Since to retrieve data only if needed.
> The same timestamp can also be used with If-Unmodified-Since for edits.

If-Unmodified-Since is what set me going with this whole draft, so obviously I value that mechanism highly. The fact that it works with timestamps worries me a bit, however. 

First, the resolution is pretty low, 1s. It is quite possible that a server could process more than one edit/POST in the same second, in which case this mechanism might indicate no change has taken place when in fact it has.

Secondly, while the idea with timestamps and "unmodified since" is very convenient, it's also quite brittle. If a client ever sends an edit request to a server without using the If-Unmodified-Since, it may miss an update made by another client. The ETag mechanism does not have that issue.

If this isn't obvious, here's an example:
1. Client A sends an edit to the server If-Unmodified-Since t0. Successful. Receives a Last-Modified timestamp t1.
2. Client B sends a an edit to the server. Last-Modified timestamp on server is now t2.
3. Client A sends an edit to the server without If-Unmodified-Since. It just sets one tiny little leaf off in one corner. Successful. Received a Last-Modified timestamp t3.
4. Client A sends an edit to the server If-Unmodified-Since t3. Successful, but clobbers Client B's edit, leading to a misconfiguration, which opens a security hole.

This is because the If-Unmodified-Since uses less than or equal in its test. The ETag mechanism is not susceptible to this issue, as it uses an equality test.

Best Regards,
/jan