Re: [IPFIX] NetFlow v9 to IPFIX conversion

Brian Trammell <ietf@trammell.ch> Tue, 06 January 2015 14:26 UTC

Return-Path: <ietf@trammell.ch>
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BB4E81A700A for <ipfix@ietfa.amsl.com>; Tue, 6 Jan 2015 06:26:48 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.912
X-Spam-Level:
X-Spam-Status: No, score=-1.912 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SAbft-SPDG5c for <ipfix@ietfa.amsl.com>; Tue, 6 Jan 2015 06:26:46 -0800 (PST)
Received: from trammell.ch (trammell.ch [5.148.172.66]) by ietfa.amsl.com (Postfix) with ESMTP id C419E1A6F3F for <ipfix@ietf.org>; Tue, 6 Jan 2015 06:26:45 -0800 (PST)
Received: from [IPv6:2001:67c:10ec:2a49:8000::ce] (unknown [IPv6:2001:67c:10ec:2a49:8000::ce]) by trammell.ch (Postfix) with ESMTPSA id 567291A0183; Tue, 6 Jan 2015 15:26:14 +0100 (CET)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\))
From: Brian Trammell <ietf@trammell.ch>
In-Reply-To: <23B7BE54EACBED43957AB709C564F7B70185A1FC39@EMEA-EXCH01.corp.brocade.com>
Date: Tue, 6 Jan 2015 15:26:14 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <CE3C776B-2C96-4370-BD7C-4EC3DE73F2A0@trammell.ch>
References: <CALbOe5O0e3tw--vCrj9FkFWVvoMAb9iZaXyRYqfNFSSqQUT94w@mail.gmail.com> <23B7BE54EACBED43957AB709C564F7B70185A1FC39@EMEA-EXCH01.corp.brocade.com>
To: Paul Aitken <paitken@Brocade.com>
X-Mailer: Apple Mail (2.1993)
Archived-At: http://mailarchive.ietf.org/arch/msg/ipfix/t6wvNVq0wSNtIwAVAsyTl76EBF8
Cc: "ipfix@ietf.org" <ipfix@ietf.org>
Subject: Re: [IPFIX] NetFlow v9 to IPFIX conversion
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: IPFIX WG discussion list <ipfix.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ipfix>, <mailto:ipfix-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ipfix/>
List-Post: <mailto:ipfix@ietf.org>
List-Help: <mailto:ipfix-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ipfix>, <mailto:ipfix-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Jan 2015 14:26:48 -0000

hi Paul, Petr,

So unless I misunderstand here, what we're talking about is third party vendors who have more or less NF9-compatible exporters that are squatting on chunks of the NF9 IE number space (i.e., there are no Cisco-defined IEs for NetFlow 9 which are > 2^15).

So the first question is why aren't they using IPFIX in the first place? But alas...

It doesn't seem right to me to use Cisco's PEN for this, since even though Cisco has reserved the squatted blocks they know about to prevent interoperability problems, there is no guarantee they know about all the blocks that exist, and it's kind of unfair to ask one company to burn PEN space to correct the mistakes of others. 

I'd be in favor of asking IANA for a new PEN for this, since it really is a protocol-level incompatibility between NetFlow 9 and IPFIX, and it would be a relatively quick AD-sponsored or opsawg draft to do. (I do think it needs an RFC, so that the PEN registry can point at something to say "what is this?"; see e.g. PEN 29305 for RFC 5103). 

Yes, this has the problem that squatting blocks might collide with each other. I'm perfectly okay with that. As a matter of policy, we should not sanction the de facto creation of codepoints in public or semipublic registries through squatting, and while building a registry of squatted blocks to appropriate PENs is a more *elegant* solution, it seems like way too much effort for the community to expend to come up with a nice fix to a messy problem caused by the lazy. 

Cheers,

Brian


> On 06 Jan 2015, at 14:52, Paul Aitken <paitken@Brocade.com> wrote:
> 
> Petr, please see replies inline:
>  
>  
> From: IPFIX [mailto:ipfix-bounces@ietf.org] On Behalf Of Petr Velan
> Sent: 06 January 2015 12:03
> To: ipfix@ietf.org
> Subject: [IPFIX] NetFlow v9 to IPFIX conversion
>  
> Hello all,
> 
> I'm not sure whether this is the right place to ask, but we encountered following problem when converting NetFlow v9 messages to IPFIX.
> 
> Some vendors (I've heard of ntop) are using elements IDs large than 32767 in NetFlow v9.
>  
> [P] That’s not invalid in NFv9. RFC 3954 allows a 16-bit “Field Type” ID.
>  
>  
> When converting messages with these elements to IPFIX, they are considered to be Enterprise Numbers. To generate proper IPFIX message, we need to do one of the following:
> a) Generate a list of the elements and map them to PEN of the correct vendor. However, this would result in an attempt to cover all possible elements that anybody used in NetFlow v9. Moreover, we would still have to somehow handle the cases where the element is unknown
>  
> [P] When I was netflow–police at cisco, the guys at Plixer helped to identify non-cisco NFv9 exporters, and I allocated blocks of 500 or 1000 IDs for each them in the 50,000 – 60,000 range – effectively blacklisting those IDs so that cisco wouldn’t create duplicate IDs.  At that time there were just 5 such blocks recognized by Cisco. If we can identify a PEN for each block then it’d be simple to write a decoder. Unfortunately this method doesn’t extend to recognizing future allocations, or export IDs that weren’t known to us at the time.
>  
> BTW, you’re also supposing that the IDs < 32768 are identical and will remain so in future.
>  
>  
> b) Request a PEN for NetFlow compatibility and just add this PEN for every element that has ID larger than 32767.
> 
> [P] It’d be more accurate to use Cisco’s PEN for this ;-)  However in principal it’s a NFv9-in-IPFIX indicator.
> 
>  
> 
> Personally, I believe that the b) is more general and error-prone. Do you think, that it would be possible to dedicate whole PEN to this cause?
> 
> [P] I believe you could convince IANA to allocate an ID. However I’m not yet convinced that it’s a good idea.
> 
> P.
>  
>  
> Thank you for any opinions,
> 
> Petr Velan
>  
> _______________________________________________
> IPFIX mailing list
> IPFIX@ietf.org
> https://www.ietf.org/mailman/listinfo/ipfix