Re: [IPFIX] [SPAM I AM] Re: RFC5101: time first flow dropped and time last flow dropped

[Andrew Feren <[e-mail]>]

Yes, this makes sense.

I gave this some thought and I don't have a better suggestion.  I also 
agree that even though the flow[Start|End]* collection of IEs 
conveniently has one IE for both first and last, observationTime* is the 
more correct choice here.

-Andrew

On 09/22/2011 03:54 AM, Benoit Claise wrote:
> Hi Andrew,
>> Hi Benoit,
>>
>> I had the same thought about using
>>
>> 322    observationTimeSeconds
>> 323    observationTimeMilliseconds
>> 324    observationTimeMicroseconds
>> 325    observationTimeNanoseconds
>>
>> The problem is that you need two timestamps in the same record (first 
>> and last).  How does that work with the above?
> Excellent question. We debated this question with Paul Aitken recently.
> Let me rephrase the question slightly differently: how does the 
> collector determine that this is a "The Metering Process Reliability 
> Statistics Options Template", and understands the semantic of the two 
> counters?
> Well. Two solutions:
> 1. we duplicate all the IEs to have the exact correct semantic.
>     
> ObservationTime[packet|flow][first|last][Seconds|Milliseconds|Microseconds|Nanoseconds] 
> ... and potentially some more such as pre|post
>     This brings multiple new dimensions to IE and will lead to an 
> explosion of IEs if we want to cover all the case
> 2. the collector tests the possible combination of IE in the Options 
> Template Record.
>     For example, if the collector sees such an Options Template 
> Record, it knows that this a "The Metering Process Reliability 
> Statistics Options Template"
>    observationDomainId (scope)
>    meteringProcessId (scope)_
> _   ignoredPacketTotalCount
>    ignoredOctetTotalCount
>    observationTimeSeconds | observationTimeMilliseconds | observationTimeMicroseconds | observationTimeNanoseconds
>    observationTimeSeconds | observationTimeMilliseconds | observationTimeMicroseconds | observationTimeNanoseconds
>
> While the solution 2 is not perfect, this was the chosen track.
>
> Does it make sense?
>
> Regards, Benoit
>>
>> -Andrew
>>
>> On 09/21/2011 07:11 PM, Benoit Claise wrote:
>>> Hi Gerhard,
>>>
>>> Thanks for your reply.
>>> See in line.
>>>>
>>>> Hi Benoit,
>>>>
>>>> See comments inline.
>>>>
>>>> On 20.09.2011 16:19, Benoit Claise wrote:
>>>>> Dear all,
>>>>>
>>>>> [this email addresses 
>>>>> draft-claise-ipfix-protocol-rfc5101bis-00.txt,  TO DO -> "time 
>>>>> first flow dropped" and "time last flow dropped" inconsistency.  
>>>>> See the discussion on the list.]
>>>>>
>>>>> Paul Aitken discovered this issue.,
>>>>>
>>>>> From [RFC5101], section 4.3. The Exporting Process Reliability 
>>>>> Statistics Option Template
>>>>>
>>>>> time first flow dropped
>>>>>                         The timestamp of the first _Flow _was 
>>>>> dropped by
>>>>>                         the Metering Process.  For this timestamp, 
>>>>> any
>>>>>                         of the "flowStart" timestamp Information
>>>>>                         Elements flowStartMilliseconds,
>>>>>                         flowStartMicroseconds, 
>>>>> flowStartNanoseconds, and
>>>>>                         flowStartDeltaMicroseconds can be used.
>>>>>
>>>>> time last flow dropped
>>>>>                         The timestamp of the last _IP packet_ that 
>>>>> was
>>>>>                         ignored by the Metering Process.  For this
>>>>>                         timestamp, any of the "flowEnd" timestamp
>>>>>                         Information Elements flowEndMilliseconds,
>>>>>                         flowEndMicroseconds, flowEndNanoseconds, and
>>>>>                         flowEndDeltaMicroseconds can be used.
>>>>>
>>>>>
>>>>> Firstly, these definitions are inconsistent since the names and 
>>>>> the first definition say "flow" while the second definition says 
>>>>> "IP packet". Obviously "IP packet" != "flow" :-o
>>>>>
>>>>> Secondly, "The timestamp of the first Flow was dropped by the 
>>>>> Metering Process." is bad English: at least it's missing "that".
>>>>>
>>>>> Thirdly, the section 4.2 doesn't take into account a metering 
>>>>> process id. Indeed, what if we have multiple caches on the exporter?
>>>>>
>>>>> Here is a proposal for 4.2 and 4.3. The underlined parts are new
>>>>>
>>>>>
>>>>>       4.2. The Metering Process Reliability Statistics Option Template
>>>>>
>>>>>
>>>>>
>>>>>     The Metering Process Reliability Option Template specifies the
>>>>>     structure of a Data Record for reporting lack of reliability in the
>>>>>     Metering Process.  It SHOULD contain the following Information
>>>>>     Elements that are defined in [RFC5102  <http://tools.ietf.org/html/rfc5102>]:
>>>>>
>>>>>     observationDomainId
>>>>>                             An identifier of an Observation Domain that
>>>>>                             is locally unique to the Exporting Process.
>>>>>                             This Information Element MUST be defined as a
>>>>>                             Scope Field.
>>>>>
>>>>>
>>>>> _    meteringProcessId (*)
>>>>>                             The identifier of the Metering Process for
>>>>>                             which lack of reliability is reported.  There
>>>>>                             This Information Element MUST be defined as a
>>>>>                             Scope Field._
>>>>>
>>>>>     ignoredPacketTotalCount
>>>>>                             The total number of IP packets that the
>>>>>                             Metering Process did not process.
>>>>>
>>>>>     ignoredOctetTotalCount
>>>>>                             The total number of octets in observed IP
>>>>>                             packets that the Metering Process did not
>>>>>                             process.
>>>>>
>>>>>     time first_packet_ignored
>>>>>                             The timestamp of the first IP packet that was
>>>>>                             ignored by the Metering Process.  For this
>>>>>                             timestamp, any of the "flowStart" timestamp
>>>>>                             Information Elements flowStartMilliseconds,
>>>>>                             flowStartMicroseconds, flowStartNanoseconds,
>>>>>                             and flowStartDeltaMicroseconds can be used.
>>>>>
>>>>>     time last_packet_ignored
>>>>>                             The timestamp of the last IP packet that was
>>>>>                             ignored by the Metering Process.  For this
>>>>>                             timestamp, any of the "flowEnd" timestamp
>>>>>                             Information Elements flowEndMilliseconds,
>>>>>                             flowEndMicroseconds, flowEndNanoseconds, and
>>>>>                             flowEndDeltaMicroseconds can be used.
>>>>
>>>> I do not find these IEs in RFC5102.
>>> Which ones?  flowEndMilliseconds, flowEndMicroseconds, 
>>> flowEndNanoseconds, and flowEndDeltaMicroseconds
>>> There are in http://www.iana.org/assignments/ipfix/ipfix.xml
>>> However, re-reading
>>>     time first_packet_ignored
>>>                             The timestamp of the first IP packet that was
>>>                             ignored by the Metering Process.  For this
>>>                             timestamp, any of the "flowStart" timestamp
>>>                             Information Elements flowStartMilliseconds,
>>>                             flowStartMicroseconds, flowStartNanoseconds,
>>>                             and flowStartDeltaMicroseconds can be used.
>>>
>>>     time last_packet_ignored
>>>                             The timestamp of the last IP packet that was
>>>                             ignored by the Metering Process.  For this
>>>                             timestamp, any of the "flowEnd" timestamp
>>>                             Information Elements flowEndMilliseconds,
>>>                             flowEndMicroseconds, flowEndNanoseconds, and
>>>                             flowEndDeltaMicroseconds can be used.
>>> And http://www.iana.org/assignments/ipfix/ipfix.xml for the 
>>> "flowEnd" timestamp IE,
>>> I now believe that we should be using this series of IEs
>>>
>>>     322    observationTimeSeconds
>>>     323    observationTimeMilliseconds
>>>     324    observationTimeMicroseconds
>>>     325    observationTimeNanoseconds
>>>
>>>
>>>>
>>>>>       4.3. The Exporting Process Reliability Statistics Option
>>>>>       Template
>>>>>
>>>>>
>>>>>     The Exporting Process Reliability Option Template specifies the
>>>>>     structure of a Data Record for reporting lack of reliability in the
>>>>>     Exporting process.  It SHOULD contain the following Information
>>>>>     Elements that are defined in [RFC5102  <http://tools.ietf.org/html/rfc5102>]:
>>>>>
>>>>>     Exporting Process ID
>>>>>                          The identifier of the Exporting Process for
>>>>>                          which lack of reliability is reported.  There
>>>>>                          are three Information Elements specified in
>>>>>                          [RFC5102  <http://tools.ietf.org/html/rfc5102>] that can be used for this purpose:
>>>>>                          exporterIPv4Address, exporterIPv6Address, or
>>>>>                          exportingProcessId.  This Information Element
>>>>>                          MUST be defined as a Scope Field.
>>>>>
>>>>>     notSentFlowTotalCount
>>>>>                          The total number of Flows that were generated by
>>>>>                          the Metering Process and dropped by the Metering
>>>>>                          Process or by the Exporting Process instead of
>>>>>                          being sent to the Collecting Process.
>>>>>
>>>>>     notSentPacketTotalCount
>>>>>                          The total number of packets in Flow Records that
>>>>>                          were generated by the Metering Process and
>>>>>                          dropped by the Metering Process or by the
>>>>>                          Exporting Process instead of being sent to the
>>>>>                          Collecting Process.
>>>>>
>>>>>     notSentOctetTotalCount
>>>>>                          The total number of octets in packets in Flow
>>>>>                          Records that were generated by the Metering
>>>>>                          Process and dropped by the Metering Process or
>>>>>                          by the Exporting Process instead of being sent
>>>>>                          to the Collecting Process.
>>>>>
>>>>>     time first flow dropped
>>>>>    _                       The time at which the first Flow Record was dropped by
>>>>>                          the__Exporting Process._   For this timestamp, any
>>>>>                          of the "flowStart" timestamp Information
>>>>>                          Elements flowStartMilliseconds,
>>>>>                          flowStartMicroseconds, flowStartNanoseconds, and
>>>>>                          flowStartDeltaMicroseconds can be used.
>>>>>
>>>>>     time last flow dropped
>>>>>                          _The time at which the last Flow Record was
>>>>>                          dropped by the Exporting Process_.  For this
>>>>>                          timestamp, any of the "flowEnd" timestamp
>>>>>                          Information Elements flowEndMilliseconds,
>>>>>                          flowEndMicroseconds, flowEndNanoseconds, and
>>>>>                          flowEndDeltaMicroseconds can be used.
>>>>
>>>> Again, I do not find these IEs in RFC5102.
>>> Same remark, I believe we should be using for the two previous IEs
>>>
>>>     322    observationTimeSeconds
>>>     323    observationTimeMilliseconds
>>>     324    observationTimeMicroseconds
>>>     325    observationTimeNanoseconds
>>>
>>> Regards, Benoit.
>>>>
>>>>>     The Exporting Process SHOULD export the Data Record specified by the
>>>>>     Exporting Process Reliability Statistics Option Template on a regular
>>>>>     basis or based on some export policy.  This periodicity or export
>>>>>     policy SHOULD be configurable.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> (*) regarding the meteringProcessId, we were hesitating between 
>>>>> the meteringProcessId and the cache id
>>>>> 1. there is a meteringProcessId in IPFIX IANA while there is no 
>>>>> cache id
>>>>
>>>> meteringProcessId should do the job.
>>>>
>>>>> 2. if the IPFIX exporter is configured from [IPFIX-CONF], what 
>>>>> should be the value in meteringProcessId?
>>>>>     [IPFIX-CONF] doesn't mention the meteringProcessId, but a 
>>>>> cache name.
>>>>>     From figure 1 and 2, it seems that there is a one to one 
>>>>> matching between the cache and the metering process.
>>>>>     If this is the case, a solution could be to add a 
>>>>> meteringProcess inside the cache in the figure 12 in [IPFIX-CONF] 
>>>>> below
>>>>
>>>> See reply to Pauls mail.
>>>>
>>>> Regards,
>>>> Gerhard
>>>>
>>>>>
>>>>>       4.3. Cache Class
>>>>>
>>>>>
>>>>>      +-----------------------------------+
>>>>>      | Cache                             |
>>>>>      +-----------------------------------+        1 +------------------+
>>>>>      | name                              |<>--------| immediateCache/  |
>>>>>      | dataRecords {readOnly}            |          | timeoutCache/    |
>>>>>      | cacheDiscontinuityTime {readOnly} |          | naturalCache/    |
>>>>>      |                                   |          | permanentCache   |
>>>>>      |                                   |          +------------------+
>>>>>      |                                   |
>>>>>      |                                   |     0..* +------------------+
>>>>>      |                                   |--------->| ExportingProcess |
>>>>>      +-----------------------------------+          +------------------+
>>>>>
>>>>>                            Figure 12: Cache class
>>>>> What do you think? Do you see another way?
>>>>>
>>>>> Regards, Paul & Benoit.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> IPFIX mailing list
>>>>> [e-mail]
>>>>> https://www.ietf.org/mailman/listinfo/ipfix
>>>
>>>
>>> _______________________________________________
>>> IPFIX mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/ipfix
>>
>>
>>
>> _______________________________________________
>> IPFIX mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/ipfix
>