Re: [lmap] New Version Notification for draft-starkcarey-lmap-protocol-criteria-00.txt

["Carey, Timothy (Timothy)" <[e-mail]>]

Joan and Barbara,

My comments to Barbara's inline <TAC>

BR,
Tim

-----Original Message-----
From: STARK, BARBARA H [mailto:[e-mail]] 
Sent: Tuesday, January 27, 2015 9:46 AM
To: Joan Luciani; 'Romascanu, Dan (Dan)'; [e-mail]
Subject: Re: [lmap] New Version Notification for draft-starkcarey-lmap-protocol-criteria-00.txt

Thanks for the comments. Replies in-line.
Barbara

> Section 2.0
> Would it be possible to include a list of protocols that currently meet
> these criteria?   Not necessarily an inclusive list, but something like:
> Although this is not an complete list, examples of protocols that meet 
> these criteria are:...then list a few examples?

I believe the idea is that other people in the WG will propose protocols that they believe meet the mandatory criteria and that do a good job with the comparative criteria. The WG would then focus itself on assessing just those protocols. I believe it would be inappropriate for the two of us who initially created this list of criteria to try to provide examples of protocols, unless we are actually proposing those protocols for LMAP consideration. There has been a call for protocol proposals into LMAP. This draft is intended purely as a list of criteria for evaluating whatever protocols are proposed. Effort spent discussing and evaluating whether and how well some protocol satisfies the criteria would be a waste of effort (IMO) if that protocol is not being proposed by someone for consideration.

> Section 2.1
> The sentence which starts:  "Note that although..." is confusing to 
> me.  It seems to be saying, follow these criteria, but if it is not in
> the protocol, that's okay.   Is this what is meant?
...
> Section 2.2  Same comment as above with regard to the sentence that 
> starts "Note that although..."

No. The protocol (that LMAP WG selects) MUST define how the criterion is achieved in the context of that protocol. It is not required that the protocol mandate that all implementations do it. It would be possible for LMAP (if LMAP selected the protocol) to mandate that LMAP implementations do it (or do it in certain cases).

> CP-MUST-1 and CP-MUST-2, suggest saying "securely established", 
> instead of just "established".  The word "must" is used, and so was 
> wondering if this is "MUST" as outlined in section 1.1 (using key words)?

Yes, I need to capitalize all such normative language in the next revision. Thx.
<TAC> Actually we saw CP-MUST-3 as providing the "secure" session. What we want to make sure of is that a secure session capability is possible but again we would let LMAP WG determine if a implementation is mandated to use a secure session through some normative text in the Framework, IM or Protocols drafts.
 
> Some of these CP-DIFF list items seem to be information that the 
> protocol needs to make available to the operator  (CP-DIFF-1, CP-DIFF-2, CP-DIFF-5).
> Maybe these items (data) should be a sub-category?  

This list of criteria is not intended for operator use. It is intended for use by the LMAP WG in assessing and comparing protocols submitted to LMAP WG for consideration as the LMAP WG selected and recommended protocol. Some operators may choose not to make use of the protocol recommended by LMAP WG. If that is the case, then those operators will have their own set of evaluation criteria and methodology for evaluating. I do not believe that helping operators do their own evaluation is in scope of LMAP WG. I do not believe there is any intention of this draft ever achieving RFC status. It is for internal WG use only. If there are criteria listed that are not appropriate for WG use in protocol assessment, then these criteria need to be removed.

> (Would some consideration about rate limit be appropriate?)

It may indeed be appropriate. What do others think? Would anyone care to propose a criterion around rate limiting?
<TAC> Yes I can see asking if there are inherent rate limiting (throttling) capabilities in a protocol for comparative purposes. The actual requirements for what is means to rate limit would have to be developed in the protocol draft. My guess is that this feature would be something that needs added-onto any specific base protocol but you never know.
 
> CP-DIFF-8 If possible, could examples be given?

Yes. If others think it is useful, I could include a list of examples, such as "(e.g., test tools, plugfests, certification programs, reference implementations)".
<TAC> I would think examples are useful - the examples seem to clarify by what we meant by interoperable.
 
> CP-DIFF-11  Would like to see this mandatory.  To ask a related 
> question is there a benefit from not having version number?  If so, what is that benefit?
>
> CP-DIFF-12 is a subquestion of CP-DIFF-11.  Could the be combined with 
> CP- DIFF-11?  My understanding is that these protocols are 
> standardized (or on the standards track), doesn't the standards 
> process provide a requirement for versioning? Is that adequate for 
> protocol numbering or is something else wanted by LMAP?

What do others think? I have no strong opinion.
<TAC> Hmmm - if we are saying a protocol will not be considered for selection if it does not handle elements with different versions - then yes we need to make this mandatory. Frankly I would elevate this to a requirement.

<TAC> On CP-DIFF-12 - Versioning and Extensibility are 2 separate things. I might be able to live with a protocol that doesn't provide a mechanism for vendor extension but I couldn't live with a protocol that didn't allow for different versions.

> Section 3.1
> RP-MUST-1 and RP-MUST-2  Is this "must" or "MUST" as outlined in the 
> key words section?

Yes, I need to capitalize all such normative language in the next revision. Thx.
 
> RP-DIFF-4 Personally, would like to see compression mandatory 
> depending on the amount of data.

What do others think? I have no strong opinion.
<TAC> Again this is more of a requirement that would need to be placed on a protocol; like versioning. We aren't creating requirements for a protocol but for comparing and selecting protocols. So again if we are saying we will not select a protocol that doesn't provide a compression capability then it should be a requirement not a comparative criteria. I don't think compression elevates to that "status". It is important though.
 
> RP-DIFF-5 Is this a MUST? ( as in MUST specify the bytes of overhead
> required...)
> 
> RP-DIFF-6 What determines "widely used"?  I am not certain that I see 
> this as a criterion. While maybe a focus initially, do not think this 
> needs to be a criterion long-term (just my opinion) but yes, I think 
> that LMAP would like to focus on protocols that are widely used initially.

LMAP WG needs to determine "widely used" when it compares different protocol proposals in the very near future. How much weight LMAP wishes to give this criterion is up to LMAP. These criteria are not expected to be long-term, as they are not expected to be maintained once LMAP has made its near-future protocol selection.
 
> RP-DIFF-7 Who determines interoperability? Are folks envisioning 
> independent interoperability testing?

I can provide the same list of examples as I suggested for the Controller protocol: "(e.g., test tools, plugfests, certification programs, reference implementations)". LMAP WG would need to decide whether it is likely that implementations of a protocol will be interoperable, based on the available mechanisms (and any known history regarding interoperability). Since these criteria are strictly for use by LMAP WG in its near-future protocol selection exercise, LMAP WG will make all determinations for proposed protocols. All are welcome to participate in LMAP WG to help make these determinations.
 
> RP-DIFF-10 and RP-DIFF-11  same comments as for the CP-DIFF-11 and
> CP-DIFF-12 above.

What do others think? I have no strong opinion.
<TAC> I would think versioning is a requirement; extensibility is comparative.