Re: [IPFIX] Options Templates, Templates, and related Data Sets sent in separate transport sessions

[Garri Djavadyan <[e-mail]>]

Paul,

Thank you so much for your reply.

Indeed, a source address translator in front of several Exporters that
use two Transport UDP Sessions to represent one IPFIX export stream,
have conflicting Observation Domain IDs, and different samplingInterval
values in the Data Records would be a nightmare for a Collecting
Process that treats multiple Transport Sessions sourced from the same
IP as the same export stream.

It seems I did not pay enough attention to the following sentence in
the section 3.1 of the RFC7011:

   Collecting Processes SHOULD use the Transport Session and the
   Observation Domain ID field to separate different export streams
   originating from the same Exporter.
   

I think a definition of the Export Stream in the Terminology (2)
section of the RFC7011 could also help avoid doubts, saying something
like: 

   A Transport Session carrying related Template Records, Options
   Template Records, and Data Records.


Now it is clear that we faced a bug that needs to be reported to the
respective vendor.

Thank you.

Kind regards,
Garri Djavadyan



On Mon, 2024-02-05 at 10:17 +0000, Aitken, Paul wrote:
> Garri,
> 
>  From the receiver's point of view, the different transport sessions
> are 
> independent export streams which could come from different Exporters.
> The receiver can't tell whether these come from one exporter or two.
> The 
> data in the "options" Export Stream cannot be scoped to data in the 
> other stream as Template IDs are unique per Transport Session and the
> Observation Domain is locally unique to the Exporting Process and 
> multiple EPs may be in play.
> 
> Only SCTP provides multiple streams; TPC and UDP do not. See section
> 6.1 
> of RFC 5153.
> 
> P.
> 
> 
> On 03/02/24 14:38, Garri Djavadyan wrote:
> > Hi IPFIX experts,
> > 
> > 
> > Recently, my colleagues and I faced an IPFIX exporter
> > implementation
> > that sends IPFIX messages with the Template and related Data Sets
> > in
> > one transport UDP session and the Options Template and related Data
> > Sets in another transport UDP session (sourced from a different UDP
> > port) to the same IPFIX collector port.
> > 
> > While trying to understand this behaviour, I checked RFC7011 and a
> > few
> > related RFCs but could not find any notes suggesting use cases when
> > it
> > might be necessary. Also, I could not find any notes prohibiting
> > this.
> > 
> > However, we know of at least one use case when sending the
> > aforementioned 4 IPFIX messages over 2 separate transport sessions
> > is
> > undesirable. For example, when multiple self-sufficient IPFIX
> > collectors are listening on the same UDP socket using the socket
> > option
> > SO_REUSEPORT (for horizontal scaling), the kernel-level load-
> > balancing
> > algorithm can distribute 2 related sessions of the same exporter to
> > different processes. As a result, the process receiving the Flow
> > Records, cannot process them properly without the related Options.
> > In
> > theory, it can also happen in the environments where layer-4 load-
> > balancers are used to distribute transport UDP sessions between
> > IPFIX
> > collecting hosts.
> > 
> > Considering this, could you please help answer the following
> > question?
> > 
> > Did I interpret the IPFIX RFCs correctly and the mentioned
> > exporter's
> > behaviour is a compliant implementation?
> > 
> > There is a note in the "Template Management" section 8 of RFC7011
> > saying:
> > 
> >     Options Templates and Templates that are related or
> > interdependent
> >     (e.g., by sharing common properties as described in [RFC5473])
> >     SHOULD be sent together in the same IPFIX Message.
> > 
> > but the Templates and the Options Templates in the case that I
> > mentioned are not interdependant by the common properties per
> > RFC5473.
> > 
> > 
> > 
> > Thank you in advance!
> > 
> > 
> > Kind regards,
> > Garri Djavadyan
> >   
> > 
> > _______________________________________________
> > IPFIX mailing list
> > [e-mail]
> > https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ipfix__;!!OSsGDw!J3DauxNGB6V13gsv8HRy629ddHwNTtJFeE5njPIdVQCVssXh2upsaiMkWBlmd8llnYXke9dEef0WZvmNag$
> >  [ietf[.]org]