Re: [IPFIX] NetFlow v9 to IPFIX conversion

Gerhard Muenz <muenz@net.in.tum.de> Thu, 02 April 2015 20:36 UTC

Return-Path: <muenz@net.in.tum.de>
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 091891A1A63 for <ipfix@ietfa.amsl.com>; Thu, 2 Apr 2015 13:36:55 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.95
X-Spam-Level:
X-Spam-Status: No, score=-1.95 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, HELO_EQ_DE=0.35, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 22sUjyR8eh9u for <ipfix@ietfa.amsl.com>; Thu, 2 Apr 2015 13:36:48 -0700 (PDT)
Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 223AB1A1A5A for <ipfix@ietf.org>; Thu, 2 Apr 2015 13:36:48 -0700 (PDT)
Received: from [192.168.2.26] (f053233063.adsl.alicedsl.de [78.53.233.63]) by mail.net.in.tum.de (Postfix) with ESMTPSA id 3A6BA191C88C; Thu, 2 Apr 2015 22:36:45 +0200 (CEST)
Message-ID: <551DA857.2050104@net.in.tum.de>
Date: Thu, 02 Apr 2015 22:36:39 +0200
From: Gerhard Muenz <muenz@net.in.tum.de>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0
MIME-Version: 1.0
To: Petr Velan <petr.velan@cesnet.cz>
References: <CALbOe5O0e3tw--vCrj9FkFWVvoMAb9iZaXyRYqfNFSSqQUT94w@mail.gmail.com> <54AC4097.1050602@plixer.com> <CALbOe5M8VtTLANGZDUG=bQH-z6eKLK7ckTPTUY0AueX_ioUs1Q@mail.gmail.com>
In-Reply-To: <CALbOe5M8VtTLANGZDUG=bQH-z6eKLK7ckTPTUY0AueX_ioUs1Q@mail.gmail.com>
Content-Type: multipart/alternative; boundary="------------080809030505030005090800"
Archived-At: <http://mailarchive.ietf.org/arch/msg/ipfix/eK3WTih668oPbLj8ZxcLKD94IlQ>
Cc: ipfix@ietf.org
Subject: Re: [IPFIX] NetFlow v9 to IPFIX conversion
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: IPFIX WG discussion list <ipfix.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ipfix>, <mailto:ipfix-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ipfix/>
List-Post: <mailto:ipfix@ietf.org>
List-Help: <mailto:ipfix-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ipfix>, <mailto:ipfix-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 02 Apr 2015 20:36:55 -0000

Hi Petr,

I do not understand the use case of the new PEN that you suggest.

It does not make sense to register a PEN for an ID space that is not 
centrally managed in a kind of registry because uniqueness of ID usage 
must be ensured. Usually, the owner of a PEN maintains such a registry.

Is there a registry that lists all vendor specific Netflow 9 Field Types?
If not, how can you be sure that there are no collisions in the ID space?

I think that you need to find and use the PENs of the vendors that have 
defined the additional Netflow 9 Field Types IEs, regardless of whether 
the IDs are below or above 2^15. Unfortunately, there is no PEN for 
general experimental use (at least I have not found any) that you could 
use as a fallback if you do not find an appropriate vendor PEN.

If you want to use your own non-standard IEs, then you should use the 
PEN of your organization:

8057
   CESNET
     CESNET masters team
       masters&cesnet.cz

Maybe, you can also use this PEN to map Field Types for which you do not 
find a vendor.

Regards,
Gerhard


On 26.03.2015 08:14, Petr Velan wrote:
> Hi Andrew, all,
>
> thank you for your explanation regarding nprobe.
>
> However, we also need a fallback for unknown exporters with IEs > 
> 2^15. The generic requests for PENs need organization name, contact 
> name and email address. I can try to request the PEN for NetFlow v9 
> compatibility myself, but I'd like it to be more public. Therefore, I 
> suggest to complete the request with something like:
> *Organization Name*: NetFlow v9 to IPFIX
> *Contact Name*: IPFIX WG
> *Contact E-Mail: *ipfix@ietf.org <mailto:ipfix@ietf.org>
>
> This is just a first proposal to get things moving, please add your 
> thoughts. Once the PEN is granted, we can move forward and explain its 
> purpose in a short RFC.
>
> Petr
>
> On Tue, Jan 6, 2015 at 9:07 PM, Andrew Feren <andrewf@plixer.com 
> <mailto:andrewf@plixer.com>> wrote:
>
>     Hi Petr,
>
>     On 01/06/2015 07:03 AM, Petr Velan wrote:
>>     Hello all,
>>
>>     I'm not sure whether this is the right place to ask, but we
>>     encountered following problem when converting NetFlow v9 messages
>>     to IPFIX.
>>
>>     Some vendors (I've heard of ntop) are using elements IDs large
>>     than 32767 in NetFlow v9. When converting messages with these
>>     elements to IPFIX, they are considered to be Enterprise Numbers.
>>     To generate proper IPFIX message, we need to do one of the following:
>>     a) Generate a list of the elements and map them to PEN of the
>>     correct vendor. However, this would result in an attempt to cover
>>     all possible elements that anybody used in NetFlow v9. Moreover,
>>     we would still have to somehow handle the cases where the element
>>     is unknown
>     This should help with ntop/nprobe
>
>     Recent versions of nprobe (since version 5.5.5 I think) all use
>     the following mapping.
>
>     PEN = 35632 and IPFIXID = (v9ID - 57472)
>
>     For example, one v9 IE that nprobe exports is MYSQL_SERVER_VERSION
>     57667.  The IPFIX equivalent would be
>     MYSQL_SERVER_VERSION(35632/195).
>
>     The nprobe docs have a complete list.
>
>     Older versions of nprobe (pre ~2010) use IEs not in RFC 3954, but
>     later allocated in IANA.  There is no good way to convert those v9
>     exports to IPFIX.
>
>     -Andrew
>
>
>>     b) Request a PEN for NetFlow compatibility and just add this PEN
>>     for every element that has ID larger than 32767.
>>
>>     Personally, I believe that the b) is more general and
>>     error-prone. Do you think, that it would be possible to dedicate
>>     whole PEN to this cause?
>>
>>     Thank you for any opinions,
>>
>>     Petr Velan
>>
>>
>>
>>     _______________________________________________
>>     IPFIX mailing list
>>     IPFIX@ietf.org  <mailto:IPFIX@ietf.org>
>>     https://www.ietf.org/mailman/listinfo/ipfix
>
>
>
>
> _______________________________________________
> IPFIX mailing list
> IPFIX@ietf.org
> https://www.ietf.org/mailman/listinfo/ipfix