Re: [IPFIX] NetFlow v9 to IPFIX conversion

Brian Trammell <ietf@trammell.ch> Tue, 06 January 2015 15:34 UTC

Return-Path: <ietf@trammell.ch>
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1F5A91A8873 for <ipfix@ietfa.amsl.com>; Tue, 6 Jan 2015 07:34:34 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.912
X-Spam-Level:
X-Spam-Status: No, score=-1.912 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T3dGuUMQozmC for <ipfix@ietfa.amsl.com>; Tue, 6 Jan 2015 07:34:29 -0800 (PST)
Received: from trammell.ch (trammell.ch [5.148.172.66]) by ietfa.amsl.com (Postfix) with ESMTP id C70B91A1A67 for <ipfix@ietf.org>; Tue, 6 Jan 2015 07:34:19 -0800 (PST)
Received: from pb-10243.ethz.ch (pb-10243.ethz.ch [82.130.102.152]) by trammell.ch (Postfix) with ESMTPSA id 130641A0920; Tue, 6 Jan 2015 16:33:47 +0100 (CET)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\))
From: Brian Trammell <ietf@trammell.ch>
In-Reply-To: <23B7BE54EACBED43957AB709C564F7B70185A1FCCD@EMEA-EXCH01.corp.brocade.com>
Date: Tue, 6 Jan 2015 16:33:47 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <2EEDA971-A002-436B-9BDF-4941E4883925@trammell.ch>
References: <CALbOe5O0e3tw--vCrj9FkFWVvoMAb9iZaXyRYqfNFSSqQUT94w@mail.gmail.com> <23B7BE54EACBED43957AB709C564F7B70185A1FC39@EMEA-EXCH01.corp.brocade.com> <CE3C776B-2C96-4370-BD7C-4EC3DE73F2A0@trammell.ch> <23B7BE54EACBED43957AB709C564F7B70185A1FCCD@EMEA-EXCH01.corp.brocade.com>
To: Paul Aitken <paitken@Brocade.com>
X-Mailer: Apple Mail (2.1993)
Archived-At: http://mailarchive.ietf.org/arch/msg/ipfix/rWOpETfNVBkcyIESj3DUT6as7Ws
Cc: "ipfix@ietf.org" <ipfix@ietf.org>
Subject: Re: [IPFIX] NetFlow v9 to IPFIX conversion
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: IPFIX WG discussion list <ipfix.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ipfix>, <mailto:ipfix-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ipfix/>
List-Post: <mailto:ipfix@ietf.org>
List-Help: <mailto:ipfix-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ipfix>, <mailto:ipfix-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Jan 2015 15:34:34 -0000

hi Paul,

> On 06 Jan 2015, at 16:31, Paul Aitken <paitken@Brocade.com> wrote:
> 
> Brian,
> 
> 
>> -----Original Message-----
>> From: Brian Trammell [mailto:ietf@trammell.ch]
>> Sent: 06 January 2015 14:26
>> To: Paul Aitken
>> Cc: Petr Velan; ipfix@ietf.org
>> Subject: Re: [IPFIX] NetFlow v9 to IPFIX conversion
>> 
>> hi Paul, Petr,
>> 
>> So unless I misunderstand here, what we're talking about is third party vendors
>> who have more or less NF9-compatible exporters that are squatting on chunks
>> of the NF9 IE number space
> 
> Correct - though "squatting" might not be quite the right word. These IDs were allocated with cisco's knowledge in order to prevent NFv9 ID clashes.
> 
> 
>> (i.e., there are no Cisco-defined IEs for NetFlow 9 which are > 2^15).
> 
> I believe there are many Cisco IEs > 32768.

Ah, okay. Then, on review... well, while maybe I'd tone down the cynicism a little bit, everything I say below about my opinion stands. :)

Cheers,

Brian

>> So the first question is why aren't they using IPFIX in the first place? But alas...
> 
> Doubtless it was easier to use some NFv9 IDs than to upgrade exporters and collectors to IPFIX.
> 
> 
>> It doesn't seem right to me to use Cisco's PEN for this, since even though Cisco
>> has reserved the squatted blocks they know about to prevent interoperability
>> problems, there is no guarantee they know about all the blocks that exist, and
>> it's kind of unfair to ask one company to burn PEN space to correct the mistakes
>> of others.
> 
> +1
> 
> 
>> I'd be in favor of asking IANA for a new PEN for this, since it really is a protocol-
>> level incompatibility between NetFlow 9 and IPFIX, and it would be a relatively
>> quick AD-sponsored or opsawg draft to do. (I do think it needs an RFC, so that
>> the PEN registry can point at something to say "what is this?"; see e.g. PEN
>> 29305 for RFC 5103).
>> 
>> Yes, this has the problem that squatting blocks might collide with each other. I'm
>> perfectly okay with that. As a matter of policy, we should not sanction the de
>> facto creation of codepoints in public or semipublic registries through squatting,
>> and while building a registry of squatted blocks to appropriate PENs is a more
>> *elegant* solution, it seems like way too much effort for the community to
>> expend to come up with a nice fix to a messy problem caused by the lazy.
> 
> As far as I know, that registry currently has just 5 non-cisco entries, so it wouldn't be hard to code.
> 
> However as you say there could be other ID conflicts which aren't known about today, and a new PEN would remove the guesswork.
> 
> P.
> 
> 
>>> On 06 Jan 2015, at 14:52, Paul Aitken <paitken@Brocade.com> wrote:
>>> 
>>> Petr, please see replies inline:
>>> 
>>> 
>>> From: IPFIX [mailto:ipfix-bounces@ietf.org] On Behalf Of Petr Velan
>>> Sent: 06 January 2015 12:03
>>> To: ipfix@ietf.org
>>> Subject: [IPFIX] NetFlow v9 to IPFIX conversion
>>> 
>>> Hello all,
>>> 
>>> I'm not sure whether this is the right place to ask, but we encountered
>> following problem when converting NetFlow v9 messages to IPFIX.
>>> 
>>> Some vendors (I've heard of ntop) are using elements IDs large than 32767 in
>> NetFlow v9.
>>> 
>>> [P] That’s not invalid in NFv9. RFC 3954 allows a 16-bit “Field Type” ID.
>>> 
>>> 
>>> When converting messages with these elements to IPFIX, they are considered
>> to be Enterprise Numbers. To generate proper IPFIX message, we need to do
>> one of the following:
>>> a) Generate a list of the elements and map them to PEN of the correct vendor.
>> However, this would result in an attempt to cover all possible elements that
>> anybody used in NetFlow v9. Moreover, we would still have to somehow handle
>> the cases where the element is unknown
>>> 
>>> [P] When I was netflow–police at cisco, the guys at Plixer helped to identify
>> non-cisco NFv9 exporters, and I allocated blocks of 500 or 1000 IDs for each
>> them in the 50,000 – 60,000 range – effectively blacklisting those IDs so that
>> cisco wouldn’t create duplicate IDs.  At that time there were just 5 such blocks
>> recognized by Cisco. If we can identify a PEN for each block then it’d be simple
>> to write a decoder. Unfortunately this method doesn’t extend to recognizing
>> future allocations, or export IDs that weren’t known to us at the time.
>>> 
>>> BTW, you’re also supposing that the IDs < 32768 are identical and will remain
>> so in future.
>>> 
>>> 
>>> b) Request a PEN for NetFlow compatibility and just add this PEN for every
>> element that has ID larger than 32767.
>>> 
>>> [P] It’d be more accurate to use Cisco’s PEN for this ;-)  However in principal
>> it’s a NFv9-in-IPFIX indicator.
>>> 
>>> 
>>> 
>>> Personally, I believe that the b) is more general and error-prone. Do you think,
>> that it would be possible to dedicate whole PEN to this cause?
>>> 
>>> [P] I believe you could convince IANA to allocate an ID. However I’m not yet
>> convinced that it’s a good idea.
>>> 
>>> P.
>>> 
>>> 
>>> Thank you for any opinions,
>>> 
>>> Petr Velan
>>> 
>>> _______________________________________________
>>> IPFIX mailing list
>>> IPFIX@ietf.org
>>> https://www.ietf.org/mailman/listinfo/ipfix
>