Re: IPFIX SCTP Stream Selection [was Re: [IPFIX] Implementation Guidelines // SCTP]

Benoit, Andrew,

> I agree with Andrew. I see the per-sctp-stream draft an en extension to 
> the current IPFIX protocol.
> Note that extension will fully make sense with the possible addition of 
> streams within an existing SCTP association.
>

yes

> Now, I understand Brian/Gehrard 's suggestions to simplify the IPFIX 
> protocol, and I'm not opposed to it.
> Nevertheless, I'm concerned about this last minute quite significant 
> change. Have we spent enough time thinking about the consequences: 
> Template Withdraw Messages, Options Template, etc... By giving the full 
> flexibility, is there enough guidelines for the different cases?
> 
> My view is that a RFC with errata is confusing. But, if the protocol is 
> changed, it will delay further the RFC publication.

One of the reasons why we propose the errata is that when implementing IPFIX over SCTP, implementers start wondering why we have the constraint of sending all templates on stream 0. This happened at the last interop, and recently Gerhard has raised the same issue. It will surely happen again. Actually there seem to be agreement on the list that the requirement is unnecessary. So the errata would simplify and clarify things.
If we managed to have this errata published quickly, the new implementations would follow the new specification. 

The "confusion" coming from the existence of an errata could be reduced by adding a reference to it in the IPFIX Documents Overview sections of all the documents in the present (and future) charters...

No doubt that the protocol would be the best place to make this kind of changes. The errata seemed just a good compromise.

> Note: the IPFIX WG has taken too long to produce RFCs (all the 
> references amongst IPFIX and PSAMP documents is one of the factor) This 
> is good in the sense we could have multiple interops to test the 
> protocol, but this leads into this complex situation where we always to 
> improve the documents. If we had more classical WG ;), I guess that we 
> would have the Proposed Standard by now, and that those changes would go 
> in the Draft Standard
> 
> Note: even the protocol is changed, I think the per-stream draft still 
> makes sense, as Andrew wrote:
> 
>     If the collector knows that the per-stream
>     rules are being used then it can infer additional information when
>     packets are lost, as well as benefit from the template/data ordering

removing the stream 0 constraint (i.e. the errata) is not an alternative to your proposed extension. On the contrary, I see it as a way to reduce the confusion people might have to see an extension contraddicting a strong recommendation (the SHOULD) given in the protocol specification. Extensions on the use of SCTP (your proposal and maybe others in the future?) based on this are perfectly fine.

Regards,
Elisa

>       
> 
> Regards, Benoit.
>> Hi Brian
>>
>> The per-stream draft specifies one possible implementation of the rules
>> given in the protocol.  While it does go against some of the
>> recommendations (SHOULDs), I wouldn't say it was incompatible with the
>> protocol as it stands today and is not intended to update the protocol.
>>
>> The intended deployment is, as you say, for multiple applications using
>> a single exporting process.  If the collector knows that the per-stream
>> rules are being used then it can infer additional information when
>> packets are lost, as well as benefit from the template/data ordering.
>>
>> If your deployment pattern doesn't benefit from these additional rules,
>> then you are free to be compliant with the protocol, but not use the
>> per-stream rules.
>>
>> regards
>>
>> Andrew
>>
>>
>>
>> On Fri, 2007-05-25 at 11:21 -0400, Brian Trammell wrote:
>>   
>>> Benoit, all,
>>>
>>> I have reviewed draft-claise-ipfix-export-per-sctp-stream-00 (herein
>>> per-stream), and have the following comments. I'll restrict myself to
>>> the content of the document itself, and the implications of this
>>> approach to the IPFIX protocol and the future work of the IPFIX Working
>>> Group.
>>>
>>> First, in general, it seems to me like the per-stream approach
>>> (restricting each template to a single stream) has a single IPFIX
>>> deployment pattern in mind: multiple "applications" running over one or
>>> more Observation Domains on a single Exporting Process, such that each
>>> "application" uses a single template. I'm inferring from the importance
>>> placed, for instance, on per-template loss rates that templates are used
>>> as a proxy for "applications". If this is the case, the approach does
>>> handle that deployment pattern quite well.
>>>
>>> However, there exist other possible (and indeed, actual) deployment
>>> patterns. The one example with which I'm most familiar is our own YAF
>>> flow meter (http://tools.netsa.cert.org/yaf) which implements a single
>>> application in a single Observation Domain on a single Exporting Process
>>> and uses multiple templates for export efficiency. The per-stream
>>> approach is not at all appropriate for this deployment pattern; opening
>>> one stream for UDP uniflows and one for TCP biflows doesn't buy us much,
>>> and we don't really care about per-template loss rates in this case.
>>>
>>> I'm concerned that per-stream updates the protocol document before the
>>> protocol's even out of the RFC Editor queue, and does so in an
>>> incompatible way, additionally contradicting the implementation
>>> guidelines (which have not yet even been submitted to the IESG), AND
>>> does so in a way that supports only a single deployment pattern.
>>> Adopting this document on the standards track such that it 1. fixes
>>> stream selection rules and 2. mandates a particular deployment pattern
>>> can only serve to confuse IPFIX implementors, potentially leading them
>>> to believe that this is the only deployment pattern supported by the
>>> IPFIX protocol, which will severely limit the protocol's adoption.
>>>
>>> I agree that the stream selection rules in the IPFIX Protocol are
>>> currently broken. Indeed, I and many others attempted to change them,
>>> but the attempt was too late in the process to be effective (a MUST
>>> became a SHOULD, but even that is in my opinion too strong). It does not
>>> appear that we as a working group or a community really understand the
>>> best way to use SCTP streams. Per-stream is one way, that would appear
>>> to work very well in certain circumstances. I'm sure there are others.
>>> If we're going to update the protocol, instead of replacing one set of
>>> restrictions with another, why not relax the restrictions to allow
>>> interoperable experimentation with stream selection rules?
>>>
>>> We could easily adopt a short errata draft, much like RFC 3309 which
>>> corrected a similar issue in the SCTP protocol itself, that relaxes all
>>> the stream selection rules, allowing an Exporting Process to export
>>> Templates and Data on any streams as appropriate for its own application
>>> requirements. Elisa and I have written such a draft, which has been
>>> submitted as draft-trammell-ipfix-sctp-change-00 and will be available
>>> shortly. The draft is quite brief, four and a half pages exclusive of
>>> boilerplate, and could be adopted and submitted to correct the stream
>>> selection problem rapidly.
>>>
>>> Per-stream could then be expressed in terms of the new unrestrictive
>>> rules, as a more-restrictive specialization of the stream selection
>>> rules useful in the specific deployment pattern which it addresses. As a
>>> working group, we can then consider the per-stream draft solely on the
>>> technical merits of the approach, separate from the stream selection
>>> problem.
>>>
>>> Regards,
>>>
>>> Brian
>>>
>>> Benoit Claise wrote:
>>>     
>>>> Gerhard, all,
>>>>
>>>> What you're proposing is something that Cisco has been thinking about
>>>> for more than 6 months now.
>>>> Actually we filled in a patent on that one. Because you proposed the
>>>> idea, we worked on the creation of a new draft
>>>> http://www.ietf.org/internet-drafts/draft-claise-ipfix-export-per-sctp-stream-00.txt
>>>> fully explaining the idea
>>>> We also posted an IPR: see
>>>> https://datatracker.ietf.org/public/ipr_list.cgi.
>>>> The title of the IPR disclosure is "Cisco's Statement about IPR claimed
>>>> in draft-claise-ipfix-export-per-sctp-stream-00.txt."
>>>> Alternatively, directly look up
>>>> http://www1.ietf.org/ietf/IPR/cisco-ipr-draft-claise-ipfix-export-per-sctp-stream-00.txt
>>>>
>>>> I didn't want to answer this email before the draft and the IPR were
>>>> posted.
>>>>
>>>> Note that a new version of this draft will follow, with the
>>>> functionality to add streams in SCTP, when available in the SCTP reset
>>>> draft
>>>> (http://www.ietf.org/internet-drafts/draft-stewart-sctpstrrst-04.txt).
>>>> Once done, my personal view is that it would be a good candidate for the
>>>> new charter discussion in Chicago
>>>>
>>>> Regards, Benoit.
>>>>       
>>>>> Hi Elisa, Benoit,
>>>>>
>>>>> Boschi, Elisa wrote:
>>>>>   
>>>>>         
>>>>>>> 6.1:
>>>>>>> The citation from protocol-24 has changed !!! Now it is:
>>>>>>>
>>>>>>>    Template Sets and Option Template Sets SHOULD be sent on the SCTP
>>>>>>>    stream zero. Template Sets and Option Template Sets MUST be sent
>>>>>>>    reliably.
>>>>>>>
>>>>>>> I propose to emphasize more on this important implementation aspect since
>>>>>>> the reliable transport of Templates is one of the main advantages of SCTP!
>>>>>>> There is a side-effect:
>>>>>>>    "A simple system might use just one data stream for all the exported
>>>>>>>    data."
>>>>>>> This implies that the all IPFIX messages are sent over one reliable
>>>>>>>       
>>>>>>>             
>>>>>> stream!
>>>>>>     
>>>>>> 6.1 has changed quite a bit and the new version should address your
>>>>>> concerns... more about this in the next days
>>>>>>     
>>>>>>           
>>>>> What is the reason for separating Templates from the corresponding Data
>>>>> Sets and for sending them over a separate stream (zero)?
>>>>>
>>>>> I've been talking to Michael Tüxen, who was working on PR-SCTP. It seems
>>>>> that the sentence
>>>>>
>>>>>     Template Sets and Option Template Sets SHOULD be sent on the SCTP
>>>>>     stream zero.
>>>>>
>>>>> from the protocol draft is a relict from very early days of PR-SCTP when
>>>>> reliability options were assumed to be a per-stream setting. Actually,
>>>>> it is a per-message property, i.e. in the same stream, one message can
>>>>> be sent fully reliable and another one partially reliable. The effect of
>>>>> a stream is that the message order is preserved and that head-of-line
>>>>> blocking at the receiver does not exist between different streams.
>>>>>
>>>>> Therefore, it seems to be more appropriate to have a separate stream for
>>>>> each Template, and to send the Template as well as the corresponding
>>>>> Data Sets over this stream. This ensures that the Template arrives
>>>>> before the Data Sets. Templates would be sent in reliable messages, Data
>>>>> Sets in partially reliable messages.
>>>>>
>>>>> Am I missing the point here?
>>>>>
>>>>> Regards,
>>>>> Gerhard
>>>>>
>>>>>   
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>> _______________________________________________
>>>>> IPFIX mailing list
>>>>> IPFIX@ietf.org
>>>>> https://www1.ietf.org/mailman/listinfo/ipfix
>>>>>   
>>>>>         
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> IPFIX mailing list
>>>> IPFIX@ietf.org
>>>> https://www1.ietf.org/mailman/listinfo/ipfix
>>>>       
>>> _______________________________________________
>>> IPFIX mailing list
>>> IPFIX@ietf.org
>>> https://www1.ietf.org/mailman/listinfo/ipfix
>>>     
> 