Re: [IPFIX] RFC5101: time first flow dropped and time last flow dropped

[Benoit Claise <[e-mail]>]

Hi Andrew,
> Hi Benoit,
>
> I had the same thought about using
>
> 322    observationTimeSeconds
> 323    observationTimeMilliseconds
> 324    observationTimeMicroseconds
> 325    observationTimeNanoseconds
>
> The problem is that you need two timestamps in the same record (first 
> and last).  How does that work with the above?
Excellent question. We debated this question with Paul Aitken recently.
Let me rephrase the question slightly differently: how does the 
collector determine that this is a "The Metering Process Reliability 
Statistics Options Template", and understands the semantic of the two 
counters?
Well. Two solutions:
1. we duplicate all the IEs to have the exact correct semantic.
     
ObservationTime[packet|flow][first|last][Seconds|Milliseconds|Microseconds|Nanoseconds] 
... and potentially some more such as pre|post
     This brings multiple new dimensions to IE and will lead to an 
explosion of IEs if we want to cover all the case
2. the collector tests the possible combination of IE in the Options 
Template Record.
     For example, if the collector sees such an Options Template Record, 
it knows that this a "The Metering Process Reliability Statistics 
Options Template"

   observationDomainId (scope)
   meteringProcessId (scope)_
_   ignoredPacketTotalCount
   ignoredOctetTotalCount
   observationTimeSeconds | observationTimeMilliseconds | observationTimeMicroseconds | observationTimeNanoseconds
   observationTimeSeconds | observationTimeMilliseconds | observationTimeMicroseconds | observationTimeNanoseconds


While the solution 2 is not perfect, this was the chosen track.

Does it make sense?

Regards, Benoit
>
> -Andrew
>
> On 09/21/2011 07:11 PM, Benoit Claise wrote:
>> Hi Gerhard,
>>
>> Thanks for your reply.
>> See in line.
>>>
>>> Hi Benoit,
>>>
>>> See comments inline.
>>>
>>> On 20.09.2011 16:19, Benoit Claise wrote:
>>>> Dear all,
>>>>
>>>> [this email addresses 
>>>> draft-claise-ipfix-protocol-rfc5101bis-00.txt,  TO DO -> "time 
>>>> first flow dropped" and "time last flow dropped" inconsistency.  
>>>> See the discussion on the list.]
>>>>
>>>> Paul Aitken discovered this issue.,
>>>>
>>>> From [RFC5101], section 4.3. The Exporting Process Reliability 
>>>> Statistics Option Template
>>>>
>>>> time first flow dropped
>>>>                         The timestamp of the first _Flow _was 
>>>> dropped by
>>>>                         the Metering Process.  For this timestamp, any
>>>>                         of the "flowStart" timestamp Information
>>>>                         Elements flowStartMilliseconds,
>>>>                         flowStartMicroseconds, 
>>>> flowStartNanoseconds, and
>>>>                         flowStartDeltaMicroseconds can be used.
>>>>
>>>> time last flow dropped
>>>>                         The timestamp of the last _IP packet_ that was
>>>>                         ignored by the Metering Process.  For this
>>>>                         timestamp, any of the "flowEnd" timestamp
>>>>                         Information Elements flowEndMilliseconds,
>>>>                         flowEndMicroseconds, flowEndNanoseconds, and
>>>>                         flowEndDeltaMicroseconds can be used.
>>>>
>>>>
>>>> Firstly, these definitions are inconsistent since the names and the 
>>>> first definition say "flow" while the second definition says "IP 
>>>> packet". Obviously "IP packet" != "flow" :-o
>>>>
>>>> Secondly, "The timestamp of the first Flow was dropped by the 
>>>> Metering Process." is bad English: at least it's missing "that".
>>>>
>>>> Thirdly, the section 4.2 doesn't take into account a metering 
>>>> process id. Indeed, what if we have multiple caches on the exporter?
>>>>
>>>> Here is a proposal for 4.2 and 4.3. The underlined parts are new
>>>>
>>>>
>>>>       4.2. The Metering Process Reliability Statistics Option Template
>>>>
>>>>
>>>>
>>>>     The Metering Process Reliability Option Template specifies the
>>>>     structure of a Data Record for reporting lack of reliability in the
>>>>     Metering Process.  It SHOULD contain the following Information
>>>>     Elements that are defined in [RFC5102  <http://tools.ietf.org/html/rfc5102>]:
>>>>
>>>>     observationDomainId
>>>>                             An identifier of an Observation Domain that
>>>>                             is locally unique to the Exporting Process.
>>>>                             This Information Element MUST be defined as a
>>>>                             Scope Field.
>>>>
>>>>
>>>> _    meteringProcessId (*)
>>>>                             The identifier of the Metering Process for
>>>>                             which lack of reliability is reported.  There
>>>>                             This Information Element MUST be defined as a
>>>>                             Scope Field._
>>>>
>>>>     ignoredPacketTotalCount
>>>>                             The total number of IP packets that the
>>>>                             Metering Process did not process.
>>>>
>>>>     ignoredOctetTotalCount
>>>>                             The total number of octets in observed IP
>>>>                             packets that the Metering Process did not
>>>>                             process.
>>>>
>>>>     time first_packet_ignored
>>>>                             The timestamp of the first IP packet that was
>>>>                             ignored by the Metering Process.  For this
>>>>                             timestamp, any of the "flowStart" timestamp
>>>>                             Information Elements flowStartMilliseconds,
>>>>                             flowStartMicroseconds, flowStartNanoseconds,
>>>>                             and flowStartDeltaMicroseconds can be used.
>>>>
>>>>     time last_packet_ignored
>>>>                             The timestamp of the last IP packet that was
>>>>                             ignored by the Metering Process.  For this
>>>>                             timestamp, any of the "flowEnd" timestamp
>>>>                             Information Elements flowEndMilliseconds,
>>>>                             flowEndMicroseconds, flowEndNanoseconds, and
>>>>                             flowEndDeltaMicroseconds can be used.
>>>
>>> I do not find these IEs in RFC5102.
>> Which ones?  flowEndMilliseconds, flowEndMicroseconds, 
>> flowEndNanoseconds, and flowEndDeltaMicroseconds
>> There are in http://www.iana.org/assignments/ipfix/ipfix.xml
>> However, re-reading
>>     time first_packet_ignored
>>                             The timestamp of the first IP packet that was
>>                             ignored by the Metering Process.  For this
>>                             timestamp, any of the "flowStart" timestamp
>>                             Information Elements flowStartMilliseconds,
>>                             flowStartMicroseconds, flowStartNanoseconds,
>>                             and flowStartDeltaMicroseconds can be used.
>>
>>     time last_packet_ignored
>>                             The timestamp of the last IP packet that was
>>                             ignored by the Metering Process.  For this
>>                             timestamp, any of the "flowEnd" timestamp
>>                             Information Elements flowEndMilliseconds,
>>                             flowEndMicroseconds, flowEndNanoseconds, and
>>                             flowEndDeltaMicroseconds can be used.
>> And http://www.iana.org/assignments/ipfix/ipfix.xml for the "flowEnd" 
>> timestamp IE,
>> I now believe that we should be using this series of IEs
>>
>>     322    observationTimeSeconds
>>     323    observationTimeMilliseconds
>>     324    observationTimeMicroseconds
>>     325    observationTimeNanoseconds
>>
>>
>>>
>>>>       4.3. The Exporting Process Reliability Statistics Option Template
>>>>
>>>>
>>>>     The Exporting Process Reliability Option Template specifies the
>>>>     structure of a Data Record for reporting lack of reliability in the
>>>>     Exporting process.  It SHOULD contain the following Information
>>>>     Elements that are defined in [RFC5102  <http://tools.ietf.org/html/rfc5102>]:
>>>>
>>>>     Exporting Process ID
>>>>                          The identifier of the Exporting Process for
>>>>                          which lack of reliability is reported.  There
>>>>                          are three Information Elements specified in
>>>>                          [RFC5102  <http://tools.ietf.org/html/rfc5102>] that can be used for this purpose:
>>>>                          exporterIPv4Address, exporterIPv6Address, or
>>>>                          exportingProcessId.  This Information Element
>>>>                          MUST be defined as a Scope Field.
>>>>
>>>>     notSentFlowTotalCount
>>>>                          The total number of Flows that were generated by
>>>>                          the Metering Process and dropped by the Metering
>>>>                          Process or by the Exporting Process instead of
>>>>                          being sent to the Collecting Process.
>>>>
>>>>     notSentPacketTotalCount
>>>>                          The total number of packets in Flow Records that
>>>>                          were generated by the Metering Process and
>>>>                          dropped by the Metering Process or by the
>>>>                          Exporting Process instead of being sent to the
>>>>                          Collecting Process.
>>>>
>>>>     notSentOctetTotalCount
>>>>                          The total number of octets in packets in Flow
>>>>                          Records that were generated by the Metering
>>>>                          Process and dropped by the Metering Process or
>>>>                          by the Exporting Process instead of being sent
>>>>                          to the Collecting Process.
>>>>
>>>>     time first flow dropped
>>>>    _                       The time at which the first Flow Record was dropped by
>>>>                          the__Exporting Process._   For this timestamp, any
>>>>                          of the "flowStart" timestamp Information
>>>>                          Elements flowStartMilliseconds,
>>>>                          flowStartMicroseconds, flowStartNanoseconds, and
>>>>                          flowStartDeltaMicroseconds can be used.
>>>>
>>>>     time last flow dropped
>>>>                          _The time at which the last Flow Record was
>>>>                          dropped by the Exporting Process_.  For this
>>>>                          timestamp, any of the "flowEnd" timestamp
>>>>                          Information Elements flowEndMilliseconds,
>>>>                          flowEndMicroseconds, flowEndNanoseconds, and
>>>>                          flowEndDeltaMicroseconds can be used.
>>>
>>> Again, I do not find these IEs in RFC5102.
>> Same remark, I believe we should be using for the two previous IEs
>>
>>     322    observationTimeSeconds
>>     323    observationTimeMilliseconds
>>     324    observationTimeMicroseconds
>>     325    observationTimeNanoseconds
>>
>> Regards, Benoit.
>>>
>>>>     The Exporting Process SHOULD export the Data Record specified by the
>>>>     Exporting Process Reliability Statistics Option Template on a regular
>>>>     basis or based on some export policy.  This periodicity or export
>>>>     policy SHOULD be configurable.
>>>>
>>>>
>>>>
>>>>
>>>> (*) regarding the meteringProcessId, we were hesitating between the 
>>>> meteringProcessId and the cache id
>>>> 1. there is a meteringProcessId in IPFIX IANA while there is no 
>>>> cache id
>>>
>>> meteringProcessId should do the job.
>>>
>>>> 2. if the IPFIX exporter is configured from [IPFIX-CONF], what 
>>>> should be the value in meteringProcessId?
>>>>     [IPFIX-CONF] doesn't mention the meteringProcessId, but a cache 
>>>> name.
>>>>     From figure 1 and 2, it seems that there is a one to one 
>>>> matching between the cache and the metering process.
>>>>     If this is the case, a solution could be to add a 
>>>> meteringProcess inside the cache in the figure 12 in [IPFIX-CONF] below
>>>
>>> See reply to Pauls mail.
>>>
>>> Regards,
>>> Gerhard
>>>
>>>>
>>>>       4.3. Cache Class
>>>>
>>>>
>>>>      +-----------------------------------+
>>>>      | Cache                             |
>>>>      +-----------------------------------+        1 +------------------+
>>>>      | name                              |<>--------| immediateCache/  |
>>>>      | dataRecords {readOnly}            |          | timeoutCache/    |
>>>>      | cacheDiscontinuityTime {readOnly} |          | naturalCache/    |
>>>>      |                                   |          | permanentCache   |
>>>>      |                                   |          +------------------+
>>>>      |                                   |
>>>>      |                                   |     0..* +------------------+
>>>>      |                                   |--------->| ExportingProcess |
>>>>      +-----------------------------------+          +------------------+
>>>>
>>>>                            Figure 12: Cache class
>>>> What do you think? Do you see another way?
>>>>
>>>> Regards, Paul & Benoit.
>>>>
>>>>
>>>> _______________________________________________
>>>> IPFIX mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/ipfix
>>
>>
>> _______________________________________________
>> IPFIX mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/ipfix
>
>
>
> _______________________________________________
> IPFIX mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/ipfix