Re: [IPFIX] NetFlow v9 to IPFIX conversion

Paul Aitken <paitken@Brocade.com> Tue, 06 January 2015 13:52 UTC

Return-Path: <paitken@Brocade.com>
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BBF9A1A6F2A for <ipfix@ietfa.amsl.com>; Tue, 6 Jan 2015 05:52:57 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7HQsXavtGQkN for <ipfix@ietfa.amsl.com>; Tue, 6 Jan 2015 05:52:54 -0800 (PST)
Received: from mx0b-000f0801.pphosted.com (mx0b-000f0801.pphosted.com [IPv6:2620:100:9005:71::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D54241A6EFC for <ipfix@ietf.org>; Tue, 6 Jan 2015 05:52:53 -0800 (PST)
Received: from pps.filterd (m0000700 [127.0.0.1]) by mx0b-000f0801.pphosted.com (8.14.5/8.14.5) with SMTP id t06DZkbb003554; Tue, 6 Jan 2015 05:52:47 -0800
Received: from brmwp-exchub01.corp.brocade.com ([208.47.132.227]) by mx0b-000f0801.pphosted.com with ESMTP id 1rrb51rv3x-1 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 06 Jan 2015 05:52:46 -0800
Received: from EMEAWP-CASH02.corp.brocade.com (172.29.19.10) by BRMWP-EXCHUB01.corp.brocade.com (172.16.186.99) with Microsoft SMTP Server (TLS) id 14.3.123.3; Tue, 6 Jan 2015 06:52:45 -0700
Received: from EMEA-EXCH01.corp.brocade.com ([fe80::18c9:7b21:74fd:7e48]) by EMEAWP-CASH02.corp.brocade.com ([fe80::548c:8759:5d5e:1c0b%12]) with mapi; Tue, 6 Jan 2015 14:52:43 +0100
From: Paul Aitken <paitken@Brocade.com>
To: Petr Velan <petr.velan@cesnet.cz>, "ipfix@ietf.org" <ipfix@ietf.org>
Date: Tue, 6 Jan 2015 14:52:42 +0100
Thread-Topic: [IPFIX] NetFlow v9 to IPFIX conversion
Thread-Index: AdApqmm5o2grLEpfTj2TdBbPqaqljgAAe/iA
Message-ID: <23B7BE54EACBED43957AB709C564F7B70185A1FC39@EMEA-EXCH01.corp.brocade.com>
References: <CALbOe5O0e3tw--vCrj9FkFWVvoMAb9iZaXyRYqfNFSSqQUT94w@mail.gmail.com>
In-Reply-To: <CALbOe5O0e3tw--vCrj9FkFWVvoMAb9iZaXyRYqfNFSSqQUT94w@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: multipart/alternative; boundary="_000_23B7BE54EACBED43957AB709C564F7B70185A1FC39EMEAEXCH01cor_"
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.13.68, 1.0.33, 0.0.0000 definitions=2015-01-06_05:2015-01-06,2015-01-06,1970-01-01 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1402240000 definitions=main-1501060136
Archived-At: http://mailarchive.ietf.org/arch/msg/ipfix/zYdC6Km5Xjefj8Y4jQhquU7lXDg
Subject: Re: [IPFIX] NetFlow v9 to IPFIX conversion
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: IPFIX WG discussion list <ipfix.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ipfix>, <mailto:ipfix-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ipfix/>
List-Post: <mailto:ipfix@ietf.org>
List-Help: <mailto:ipfix-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ipfix>, <mailto:ipfix-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Jan 2015 13:52:58 -0000

Petr, please see replies inline:


From: IPFIX [mailto:ipfix-bounces@ietf.org] On Behalf Of Petr Velan
Sent: 06 January 2015 12:03
To: ipfix@ietf.org
Subject: [IPFIX] NetFlow v9 to IPFIX conversion

Hello all,
I'm not sure whether this is the right place to ask, but we encountered following problem when converting NetFlow v9 messages to IPFIX.
Some vendors (I've heard of ntop) are using elements IDs large than 32767 in NetFlow v9.

[P] That’s not invalid in NFv9. RFC 3954 allows a 16-bit “Field Type” ID.


When converting messages with these elements to IPFIX, they are considered to be Enterprise Numbers. To generate proper IPFIX message, we need to do one of the following:
a) Generate a list of the elements and map them to PEN of the correct vendor. However, this would result in an attempt to cover all possible elements that anybody used in NetFlow v9. Moreover, we would still have to somehow handle the cases where the element is unknown

[P] When I was netflow–police at cisco, the guys at Plixer helped to identify non-cisco NFv9 exporters, and I allocated blocks of 500 or 1000 IDs for each them in the 50,000 – 60,000 range – effectively blacklisting those IDs so that cisco wouldn’t create duplicate IDs.  At that time there were just 5 such blocks recognized by Cisco. If we can identify a PEN for each block then it’d be simple to write a decoder. Unfortunately this method doesn’t extend to recognizing future allocations, or export IDs that weren’t known to us at the time.

BTW, you’re also supposing that the IDs < 32768 are identical and will remain so in future.


b) Request a PEN for NetFlow compatibility and just add this PEN for every element that has ID larger than 32767.
[P] It’d be more accurate to use Cisco’s PEN for this ;-)  However in principal it’s a NFv9-in-IPFIX indicator.

Personally, I believe that the b) is more general and error-prone. Do you think, that it would be possible to dedicate whole PEN to this cause?
[P] I believe you could convince IANA to allocate an ID. However I’m not yet convinced that it’s a good idea.
P.


Thank you for any opinions,

Petr Velan