Re: [netconf] Comments on draft-jgc-netconf-privcand

[Jan Lindblad <[e-mail]>]

James,

Thanks for thinking about how these drafts may be aligned. My comments below, marked [janl].

> Hi Joe, and Jan as this is txid related), 
> And the wider working group…..
>  
> Rob and I (the authors of this draft) were talking today about the suggestion of the using the TXID to identify conflicts. 
>  
> This is how we understand the proposal might be described.
>  
> Consider the following pseudo configuration (in tree format to make it easier to read) in the running
> configuration at the start of this workflow.
>  
> +-- configure (txid: 0)
>     +-- myvpn [vpn-name=”foo”] (txid: 0)
>         +-- interface [interface-name="intf1"] (txid: 0)
>            |   +-- description "intf1_desc" (txid: 0)
>            +-- interface [interface-name="intf2"] (txid: 0)
>                +-- description "intf2_desc" (txid: 0)
>                
> And consider the following workflow (using the private-candidates style branch diagram format):
>  
>        +-------------------*  Shared candidate
>       /                     \
>      /                       ⌄
> +----+-+---+-----------------+-----------------------------> Running
>         \   \                          
>          \   \                        
>           \   +-----------------------(y)--------------> Private candidate 2
>            \   
>             +-------------------------(x)--------------> Private candidate 1
>                
> For the purposes of this email, we're going to focus just on the conflict detection when using
> txid (and I'm going to simplify the txid part a little to make it easy to read). 
>  
> The private candidate at creation time (the start of its branch) has the identical configuration as the
> initial running, including the same txid (0) on all elements.
>  
> Step 1. The user configuring the shared candidate changes the description of interface 1 and commits it.
> The txid is changed on the /configure/myvpn[vpn-name="foo"]/interface[interface-name="intf1"]/description
> path and back up the tree to the root:
>  
> +-- configure (txid: 1)
>     +-- myvpn [vpn-name=”foo”] (txid: 1)
>         +-- interface [interface-name="intf1"] (txid: 1)
>            |   +-- description "intf1_desc" (txid: 1)
>            +-- interface [interface-name="intf2"] (txid: 0)
>                +-- description "intf2_desc" (txid: 0)
>                
> Step 2. The user configuring private candidate 1 adds a description to the vpn itself and issues and
> update (at point (x)) which (assuming the default revert-on-conflict) succeeds with no conflict
> detected because the system checks the /configure/myvpn[vpn-name="foo"]/description path and finds it
> doesn't exist (so no txid) meaning there is not a conflict.  If it were to commit (which lets say right
> now that it does not) the resulting configuration would be this:
>  
> +-- configure (txid: 2)
>     +-- myvpn [vpn-name=”foo”] (txid: 2)
>         +-- description "new description" (txid: 2)
>         +-- interface [interface-name="intf1"] (txid: 1)
>            |   +-- description "intf1_desc" (txid: 1)
>            +-- interface [interface-name="intf2"] (txid: 0)
>                +-- description "intf2_desc" (txid: 0)
>                
> Step 3. Another user configuring private candidate 2 changes the description in interface 1 and issues an
> update (at point (y)) which (assuming the default revert-on-conflict) fails saying there is a conflict
> because it has checked the txid on that leaf when it started (txid: 0) against the txid on that same
> leaf in the running configuration and found that it now has a different txid (txid: 1) and as 0 != 1
> this means conflict.
>  
> Is this what you had in mind?
>  
> Are there any situations where this wouldn't be the case, or you see issues with this working? 

[janl] As far as I understand from the examples, the collision detection would happen entirely by comparing txids on leafs and leaf-list elements (probably also anydata)? I think something like this could be one basic way of leveraging the txid mechanism. An important use case not shown in the examples above is how deletion works. And we'd have to think about how p-containers should work in this context.

With this "leaf change-based" approach (assuming I got it reasonably right from the examples), clients have little control over how the collision detection happens, but don't really have to be aware of any txid values, which keeps things very simple for them. I suppose the conflict detection mechanism would be limited to only+always detect conflicts at leaf level, but perhaps that is good enough for many use cases.


[janl] Another, more elaborate way, would be to allow clients to really use the txid attributes as described in the txid draft in their edit-config (edit-data) messages. With this approach (which is already described for both shared and private candidates in the txid doc), clients would send the txid values they care about embedded in the edit payloads. This would give clients fine control over how the conflict detection/resolution happens. 

Let's say one client doesn't care at all about any conflicts in the interface description. It would therefore not send any txid value for the interface description it changed, and the change would go through regardless of any changes by others. This is similar to how edits work today for all datastores.

Another client might have a more cautious approach. It might consider any change at all by anyone else under the vpn instance foo worth flagging. It would send the last known txid for the foo instance, and therefore consider any change of that instance a collision, regardless of whether the particular leaf(s) it changed was changed by anyone else or not.

> I'm thinking of some more complicated items such as changing leaf-lists and changing metadata (annotations).

[janl] Regarding leaf-lists, I think it would make sense to treat each list entry instance like a separate leaf. That corresponds to the way lists work, and how leaf-lists are encoded in NETCONF XML and RESTCONF XML or JSON.

Regarding metadata changes, the txid document only has config true data in scope; operdata and metadata are currently not covered by this mechanism. Nothing prevents us of course from extending the txid doc for this use case as well, or to design a new mechanism in privcand that borrows/refers to some sections in txid, or come up with a completely new mechanism or document.
 
> Additionally, how would system configuration where systems currently self-provision data, and how in
> future, would items provisioned (either by copy or inheritance) from a system datastore be expected to work
> from a txid standpoint?

[janl] The exact design of the system datastore is still in the air, so it may be hard to say something conclusive at this point. From my perspective, it would seem natural that system datastore changes would appear as if an on-board client had made the changes. Such changes would show up in the system datastore, as well as perhaps in running, intended or candidate datastore(s).

> We would need to differentiate in the txid draft (and I don’t recall whether you already do Jan – sorry)
> between no txid on new items as opposed to inheriting the parents txid on creation.  If the inheritance
> approach was taken then this method of conflict detection would yield ‘false conflicts’.

[janl] Well, it's rather the other way around. Parent (ancestor) nodes in the YANG tree are marked with the new txid as their children are changed. The way I understood the examples above, only txids of "leaf-like" (value bearing nodes) would be compared, and in that case I don't think there would be any false conflicts. As noted earlier, the examples do not show deletion and p-containers, so there would be a few things to sort out with this approach, and it is possible that solutions to those issues might lead to situations with false conflicts.

I am convinced that there are good solutions to using txid that would not suffer from false conflicts. One such solution could be fully embracing the txid mechanism as specified, also in privcand settings.

> If we went forward with this approach, we would be grouping these drafts together though which would mean
> an implementation would HAVE to implement txid in order to be able to use private candidates which I am
> not sure (at this time) that I would favour.

[janl] I think the important point here is that we don't want *two* somewhat similar mechanisms for detecting collisions in a data store. 

Even if a unified mechanism would be described in the txid document, that does not necessarily imply that servers would have to implement the txid APIs/features. The privcand document could very well refer to the mechanism described in txid in a way that makes it possible to implement privcand without txid. But I am also open to other solutions to the document puzzle.

> Just to address one specific in your email Joe…… I am not sure that using txid helps at all in conflict RESOLUTION
> only in DETECTION but I suspect this is what you meant anyway but clarification would help.

[janl] While it is true that the txid does not mention conflict resolution, it does set the stage for it. Since clients declare in their edit payloads what YANG nodes they care about not being changed (or not), the conflict resolution becomes a lot simpler. Either the client's conditions for going through with the edit are fulfilled (and the transaction goes through), or there is a problem. All the middle cases, which the client would have to go through and decide if it is a real problem or not (all the while still further changes may be happening) are already taken care of. What exactly the client does once a real problem has been detected is of course out of scope from both txid and privcand.

Best Regards,
/jan


 
> From: netconf [e-mail] <mailto:[e-mail]> on behalf of James Cumming (Nokia) [e-mail] <mailto:[e-mail]>
> Date: Monday, 31 July 2023 at 09:45
> To: Joe Clarke (jclarke) jclarke=[e-mail] <mailto:jclarke=[e-mail]>, [e-mail] <mailto:[e-mail]> [e-mail] <mailto:[e-mail]>
> Subject: Re: [netconf] Comments on draft-jgc-netconf-privcand
> 
> Thanks for the comments Joe.  The authors will take a good look through them and address them either in the next version of the draft or on the list shortly.
>  
> It’s really useful to have the specific feedback and I’d encourage others to share as well.
>  
>  
>  
>  
> From: netconf [e-mail] <mailto:[e-mail]> on behalf of Joe Clarke (jclarke) jclarke=[e-mail] <mailto:jclarke=[e-mail]>
> Date: Thursday, 27 July 2023 at 14:44
> To: [e-mail] <mailto:[e-mail]> [e-mail] <mailto:[e-mail]>
> Subject: [netconf] Comments on draft-jgc-netconf-privcand
> 
> I mentioned some of this in chat (as my browser permissions were giving me a fit).  I support this work.  I think taken with the tracing and txid work it brings the possibility of more context and metadata to configuration changes.  I like the “git” aspects being pushed to the NETCONF/RESTCONF server.
>  
> I agree with Balazs that conflict resolution is important here, and I like the idea of using txid as a means to do this.  I don’t particularly like the auto-update approach, though.  I’d rather attempt a <commit> and find than an update and potential resolution is needed.
>  
> I was also going to make a similar comment to Quifang.  I would like to see the ability to delete a private candidate on commit.  If we care about the private candidate name (perhaps for tracing), I might want to do one type of change in private-add-exampleco1-vpn, commit it, and then have the candidate deleted so I can continue my session with a new private candidate for another change.
>  
> I suppose I can <delete-config> this to remove the old private candidate, but I’d like an extension to commit to do this at once (though not so much of a big deal).  I’d rather not have to disconnect and reconnect the session to perform this type of flow, though.
>  
> Joe
>  
>  
>  
>  
> _______________________________________________
> netconf mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/netconf