Re: Rtgdir telechat review of draft-ietf-anima-autonomic-control-plane-13

["Joel M. Halpern" <[e-mail]>]

Trimmed.
Top posting for clarity.  Mostly, agreement.
On the MTI Security, I will leave it to the relevant ADs to decide if 
they find what is in there sufficient.

On the identifier vs locator, I think the way you are using the terms is 
confusing.  But it is not worth continued argumentation.Some text 
elaborating along the lines you provide (retained below) would help.

I have tried reading the section on addressing inside the ACP more 
carefully.  Thank you for moving the Z bit.
I have retained the earlier discussion on this topic below, in case we 
still need it.
Reading the section, I think that the problem is more basic.

I agree that the document needs to define that we are using a ULA.  And 
because of the certificate behaviors, there is significant advantage in 
defining what ULA prefix is used.

Given that, I do not see what use there is in all the other format 
definition.  I suspect I am missing something very basic.  Why not leave 
this to the deploying operator?  It is not like it can be autonomic, 
since something has to tell the devices what format to use, and what 
numbers within some of the formats to use.  I could imagine something 
that said "in the absence of configuration, construct device addresses 
according to one specific algorithm, so as to simplify bootstrap.
But it would be a simple scheme.

Yours,
Joel

On 5/11/18 10:09 PM, Toerless Eckert wrote:
> 
> Diff:
> 
> http://tools.ietf.org/tools/rfcdiff/rfcdiff.pyht?url1=https://raw.githubusercontent.com/anima-wg/autonomic-control-plane/2ae8f47399ae0d0811cb45209186d01f9e0d3077/draft-ietf-anima-autonomic-control-plane/draft-ietf-anima-autonomic-control-plane-14.txt&url2=https://raw.githubusercontent.com/anima-wg/autonomic-control-plane/54c9c16f3444b1db13c7cb05744e543dfcdfb63b/draft-ietf-anima-autonomic-control-plane/draft-ietf-anima-autonomic-control-plane-14.txt
> 
> Latest version:
> https://raw.githubusercontent.com/anima-wg/autonomic-control-plane/54c9c16f3444b1db13c7cb05744e543dfcdfb63b/draft-ietf-anima-autonomic-control-plane/draft-ietf-anima-autonomic-control-plane-14.txt
> 
...
> Here are the things that make the ACP address an identifier to me:
> 
> - It is assigned once to the device and not meant to change over the lifetime
>    of the device.
> - It does not change when i move the node to a different location
>    and re-connect the device.
> - It is the unique part of the ACP domain certificate (other than the private key)
>    and used to identify the device (privat key is used to authenticate it,
>    but the private key may change over time, for example when renewing the certificate).
> 
> Would be happy to elaborate this in the doc more explicitly, e.g.: like written
> above.

(Context: Addressing inside the ACP.)
>> Part of my reaction to this whole section is that it looks much like the
>> mistake we made originally in defining classful IP addresses.  You are
>> trying to guess the correct use cases, and defining specific address
>> structures with specific encodings for each one.  With only a few encodings
>> available.  This seems to be a recipe for being wrong later in ways that
>> hurt.
> 
> The only equivalent to classfull addressing i see is for the fixed prefix-length
> handed out to each node, 1 bit (2 addresses) with zone addresses, 8/16 bits
> (256/64k address) for Vlong, but its NOT assignment to network segments
> as in IPv4 A/B/C but to nodes.
> 
> We didn't introduce choices lightly. There are compromises to be made when you
> want some lightweight address allocation scheme, not expecting to have complex
> address registry or management system in the backend but simple registrars.
> 
> see below for more
> 
>>>         The difference in interpretation
>>>>       is actually provided by the last bit in the upper 64.  If these are the
>>>>       same schema, then it should not need a bit to differentiate them.  They
>>>>       should be explained as a single schema.  Which would presumably result in
>>>>       the Z bit being part of the zone / subnet field.   If they are two
>>>>       different schemas, then move the differentiation to part of the schema
>>>>       identifier (making it 3 bits).
>>>
>>> We did not want to make Z part of the base schemas 6.10.2 "Type" code because
>>> that would also make the Vlong scheme one bit shorter and that would reduce
>>> virtualization space in Vlong by a factor of 2.
>>>
>>> So, yes, encoding wise, 6.10.4 manual sub addressing space was carved out
>>> of the 6.10.3 zone address space via the Z bit, but thats just an encoding aspect.
>>>
>>> Functionally, 6.10.3/6.10.4 are quite different, and therefore it is IMHO
>>> textually correct to have them in different subsections.
>>>
>>> Carving out bits like Z isn't ideal, but in between the ULA prefix,
>>> the 64 bit local part we must have for external interfaces and
>>> best utilization of bits, we tried to rather optimize address space
>>> utilization and not beauty of encoding.
>>>
>>> Having  at the end allows (as written) use of /63 instead of  2 * /64
>>> routes. I am not sure how important this would be in the future.
>>>
>>> If you feel strongly about more beauty in encoding,
>>> i can move Z to the left and remove the notion of the possible /63 route
>>> optimization possible through the placement of Z.
>>
>> Since the two different uses of the Z bit with type 00b are quite distinct,
>> trying to enable aggregation seems counter-productive.
>>
>> While I hate to push on a point you describe as aesthetics, having the Z
>> bit, occuring after the Zone / subnet-ID field, but defining whether the
>> field is a zone or a subnet, seems awkward for insufficient reason.
>>
>> I would recommend moving the Z bit up so it is after the type, and at the
>> front of each of the two sections reiterate that this subscheme is indicated
>> by the Z bit being { 0 | 1 }.
> 
> Done. (see diff).
> 
>>> I very much hope there is no "interesting" implementation aspect;
>>> nothing in the current scheme that makes implemenation harder. If you
>>> have an example that you fear, i would like to hear it.
>>>
>>> To me, we are just defining address space allocation schemes.
>>
>> I suppose likely implementations will use mask definitions that will make it
>> work.  I pity the code reviewer.
> 
> I have read the code and tested in the lab IPv6 embedded RP adressing
> and didn't feel pityfull about the job. But i weigh the overall system
> complexity. Embedded-RP simplified a lot of other parts of the
> multicast system at the cost of introducing an addressing scheme.
> I think similarily, the ACP addressing scheme simplifies the overall system
> complexity, especially registrars.
> 
>>>>       In section 6.10.5, after stating that it is not even known what the needed
>>>>       usage is for more V values
>>>
>>> Which sentence are you referring to when you say "not even known what the
>>> needed usage is" ? I can't find it, but i would like to rectify it
>>> if it actually is still there.
>>
>> The text reads "further values use via definition in future work."  In the
>> zone-id scheme, it is one bit.  For some reason, it is 8 or 16 bits here.
> 
> Thanks, changed sentence to:
> 
> V: Virtualization bit: Values 0 and 1 are assigned in the same way as in the Zone-ID sub-scheme.
> 
> The existing text, now directly after the list, explains examples
> when a device would likely need 256 or 64k addresses:
> 
> existing text:
> ..."1" bit Node-Numbers are intended for ACP nodes that are ACP edge nodes ...
> 
>>>>       the document goes on to introduce the
>>>>       complexity of classful nodes numbers, with the leading bit indicating how
>>>>       long the node-number is.  Yes, the rest of the text in that bullet tries to
>>>>       explain why you need both sizes.  It seems like needless complexity that
>>>>       begs for mistakes in implementation.
>>>
>>> I was taking the example use cases we brainstormed into account, and
>>> some of them  would require a lot more addresses in the node than others,
>>> therefore these two choices.
>>
>> See my comments above.  Trying to encode specific use cases into the address
>> format seems a problematic answer to an admittedly difficult question.
>>
>> What would break if you didn't define any of this?
> 
> One could save the 2.5 bits (Type, Z) indicating the structure of the
> address inside the address by encoding the designated format of the
> address instead in an extension field of the ACP address information in
> the certificate. We would certainly still want to have the same type
> of addresses, e.g.: 2,256,64k local addresses, option for zones and
> address to assign to ACP connect interfaces (manual addressing scheme)
> IMHO. So, at best we could save the 2.5 bits if we wanted to keep the
> functionality, which i think is all required.
> 
> I would be quite afraid of doing such a drastic change this late in
> the process because i think it will take a long time to foresee the
> resulting complexity introduced for registars and any other place
> that now can deduct the semantic from the address and would then have
> to access to what the certificate indicates in a different way (e.g.:
> access to the certificate). Note that the length of the ACP information
> field in the cert is also limited, so this type of extension might end
> up looking even uglier to be short *sigh*.
> 
> I was always thinking that this type of more flexible option would
> be a good next-step, but not a safe first step. It would make a lot
> more sense after hearing implementation/deployment experiences,
> especially to correctly support important options we may have
> overlooked in the current schemes. Just redoing what we have differently
> to save 2.5 bits isn't worth it IMHO.
> 
>>> Ultimately, we need to break through the chicken and egg problem
>>> of how to build more self-managing networks. These will require
>>> a different number of addresses inside nodes. Anything more intelligent
>>> than address space allocations would become a complex dependency
>>> making implementation/adoption even slower. I don't think that
>>> offering 2, 256, 64k addresses per node is too much flexibility.
>>> It IMHO necessary and sufficient to explore and we'll see what sticks.
>>
>> Variation is needed.  That was true of allocations in IP.  encoding the
>> variations proved a bad choice.
> 
> Let me repeat that we are not doing any network/subnet prefix encoding which is
> what IPv4 A/B/C was. And there are IMHO good examples how encoding in IPv6
> did save the day from complexity (at least all the stuff done for IPv6
> multicast, ASM/SSM, zones, Embedded-RP) was highly useful.
> 
>>>>       In section 6.11.1.11 in describing the prefix lengths, I thought that the
>>>>       point of zone addressing was to allow the use of /64 prefixes.  But it
>>>>       seems here that we will not do so ?
>>>
>>> As mentioned above: This document does not describe the routing setup for /64
>>> (or /63 with Z-bit) routing aggregation. There are multiple options,
>>> but we could not conclude on a single solution that we felt to be
>>> appropriate for this document. Instead it was only very important
>>> to carve out addressing space to support this.
>> The net effect seems to be to prohibt the very things that you went to a lot
>> of trouble to enable in your addressing design.
> 
> Maybe i do not understand your comment correctly, but i do not
> think it is correct. We just have not defined recommended mechanisms
> how to set up routing to use /64 zone routes. That is something IMHO
> better done in a followup document because all the cases where we
> did see a need for znes where really very specific to particular
> deployment styles and would not be applicable to the mayority of ACPs.
> Nevertheless, we felt those cases where important enough to reserve
> bits for them through the zone field.
> 
>> ...
>>>> ***    Section 10.3.2 paragraph 2 says that devices should change the meaning
>>>> of "admin down" to mean "down for everything except ACP / Anima".    I can
>>>> understand why, in an autonomic network, such a state is desirable.  However,
>>>> that really should be a different state from "admin down".  Operators already
>>>> understand "admin down" as meaning that this interface will not be used for any
>>>> traffic.  Changing that is fairly drastic.  "admin limited" or "admin ACP-Only"
>>>> would seem much better than changing the meaning of "admin down".  The
>>>> justification in the text seems to be a desire to prevent an operator from
>>>> doing what he intends.  that seems backwards.  (Note that the distinction
>>>> between administrative state vs operational state, aka failed, is already well
>>>> understood.)
>>>
>>> Well, this is in an informative section, so hopefully something we can agree to disagree.
>>
>> If this were truly informative, it would not belong in this document at all.
>> Changing the behavior of widely understood configuration behaviors is NOT a
>> small thing.  Changing the permissions models for widely understood
>> configuration operations is also NOT a small thing.
>>
>> My suggestion is that you remove all of this material.  If you feel it is
>> important, write an I-D for standards track on evolving the configuration
>> model for robust ANIMA.  While I think the change is a mistake, you clearly
>> have the right to cause the discussion.
>>
>> Hiding the topic in an "informational" note in the ACP document seems VERY
>> wrong.
>>
>> Also note that in your discussion below, you talk about the equivalence of
>> admin down and physical down.  Most management models I know of treat these
>> as quite distinct.  SO that is at best a red herring.
> 
> Hmm. I really only CLI derived from Cisco IOS, and there the
> interface level "shutdown" command doubles IMHO both as admin down and
> as phy down, so i would contend that this is a red herring. It actually
> does show up in diagnostic CLI as "admin down", but is also bringing down
> phy state. Therefore it is also used to diagnose phy, .e.g: shutting down
> one side of a supposed direct p2p link to verify the phy change on the
> other side. That practical experience is what the text and direction is
> based on.
> 
> Whats an example where admin/phy down are separate in your experience ?
> 
> I don't know how you define "truly informative". I simply meant
> informational as compared to standards track wrt. to not impacting
> interoperability, but also as a precursor work to give opportunity
> for gaining more experience before attempting to go standard.
> 
> If/when i find time i definitely would like to investigate how this
> could be formalized through a YANG model for ACP so that it can become standard,
> but i definitely agree that especially this part of the YANG model would
> be highly difficult to get to before any experience is gained with
> this.
> 
> I therefore would very much hesitate removing this text from the document, it was
> specifically asked for several times in the working group to understand
> how the promised reliability of the ACP against operator/contoller
> operation could be achieved. Having this in a first round (this document)
> as "informational" is IMHO a good and important thing.
> 
>> (text retained for context of further discussions.)
>>
>>>
>>> I strongly believe that the most easy way to operationalize the ACP and
>>> achieve its goals is what we have written.
>>>
>>> The historic equivalence of "down" = "admin down" = "physical down" is one of
>>> the the most fundamental problem of remote management in networks.
>>>
>>> We really need to start thinking of separate layers of (remote) management.
>>> Network services on one hand and physical infra on the other.
>>>
>>> The first thing to do this is to separate operations such as "down"
>>> into "admin down" that affects networks services and "phy down" that
>>> affects the physcial infra.
>>>
>>> If you have existing commands like "shutdown" that today do both,
>>> the safe mapping is to let them do the safe thing in the future, e.g:
>>> only "admin down", and introduce new commands for the phy operations,
>>> e.g.: "phy shutdown" or the like.
>>> And then protect those dangerous commands even further against unintentional or
>>> intentional misuse.
>>>
>>> Think of ACP/ANI also as a seamless inband version of a DCN. As an operator
>>> of the actual data network, you also didn't have any access to bring down the DCN
>>> and cut yourself off from the network you manage. YOu could only screw
>>> up that network you're meant to manage.
>>>
>>> And if your network consists of VMs in a DC, a "shutdown" on their
>>> interfaces will also not bring down physical interfaces necessary
>>> to reach the VM.
>>>
>>> In optical networks you often have an inband physcial ethernet management
>>> channel. Making/breaking that one is completely different from making/breaking
>>> your data-plane, aka: data fiber colors.
>>>
>>>> ***    The notion in 10.3.6 that the device should refuse to disable
>>>> functionality when an authorized administrator directs such seems flatly wrong.
>>>
>>> The authorization to break your own remote management connectivity
>>> would be a property of the certificate. clearly whoever enrolled
>>> the device with a certificate denying that capaility to the operator
>>> did NOT authorize the administrator to break connectivity.
>>
>> The security model change seems as problematic as the above configuration
>> change.
> 
> We have recurringly heard from customers that mistakes in managing remote
> equipment and loosing connectivity is a mayor pain point. Creating
> an operational option that limits/disables operators ability to
> impact phy level connectivity that remote management depends upon and therefore
> reducing this risk and cost factor seems to me like a very logical pah
> forward. Whether or not this should be indicated in the certificate
> or differently is definitely wide open.
> 
> I agree that none of these operational interface level changes are
> a piece of cake given how we have in the IETF not tried to define
> different access levels to configuration so far, at least not
> between some phy/infra access level and network-services access level.
> 
> As said above in my first reply that you didn't comment on, i think a
> good part of what i am looking for may happen through other means
> already, such as when a router is just a VM running on an OS. At least
> in early versions of this that i'd seen, this implicitly disabled
> the ability to modify phy state from the router VM, and of course
> complaints where raised that phy mods where necessary, but at least this
> evolution could give raise to pause and rethink HOW to permit doing this.
> 
> So i would at least like to pledge with you that it is a good thing to
> have this type of discussion in an informational way in this RFC instead
> of doing nothing or trying to go standard YANG directly as a first step.
> It did get into the spec because of the desire of the working group.
> 
>>>> Editorial / Nits:
>> .
>>>>       In the various formats in section 6.10, the lowest bit of the upper 64 bits
>>>>       is mandated to be 0.  Presumably, there is some reason for doing this.  It
>>>>       would be nice to explain it.
>>>
>>> Sorry, not clear what you're referring to. Can you give me an
>>> explicit reference ?
>>
>> I should ahve removed this note.  It was a side-effect of the presentation
>> and ordering, not a problem on its own.  Sorry.
> 
> Cheers
>      Toerless
>