Re: [IPFIX] templateLifePacket on IPFIX collector

Petr Velan <petr.velan@cesnet.cz> Mon, 02 October 2017 13:24 UTC

Return-Path: <thorgrin@gmail.com>
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CA76D132076 for <ipfix@ietfa.amsl.com>; Mon, 2 Oct 2017 06:24:21 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.398
X-Spam-Level:
X-Spam-Status: No, score=-2.398 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.199, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ca8W7wddAe0s for <ipfix@ietfa.amsl.com>; Mon, 2 Oct 2017 06:24:18 -0700 (PDT)
Received: from mail-oi0-x22c.google.com (mail-oi0-x22c.google.com [IPv6:2607:f8b0:4003:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A96E21321DF for <ipfix@ietf.org>; Mon, 2 Oct 2017 06:24:18 -0700 (PDT)
Received: by mail-oi0-x22c.google.com with SMTP id y64so8952095oia.7 for <ipfix@ietf.org>; Mon, 02 Oct 2017 06:24:18 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=/vA2D1p4oeXO8MdeqR9RSrTUUIKi88XpIp864jZHYP0=; b=tbAS3MOsI53WPOyxA0TqjF4EHsGngrmJwsX2avtSwUJjI6nUCMDy8JW1+p2ohj3piO /rsWAIUE7tyYyCQcXl1xjirZeJ/s2bTSgQR5RADpr2Q96A+3z6f1dILK7JMWmbP1aXXt BufrxJk6qpJ13H20dAbSWXkRH6quAvYRyz5FXvq1BttkxsTt+6A+uhcW0ATG7bpxlEJy GCSmyugzOdRmZIEh/akPBaNluCIOeRVFll16cFNSeB2qXtz+eQXpA3wmD3/7U9U6Cq9i qQv3Qp1udPvOR7MNCpCDREW49v/YFGeL9rPTSzltZWtvvEeZ8/limouYPfrKfMFmrZq7 ERfQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=/vA2D1p4oeXO8MdeqR9RSrTUUIKi88XpIp864jZHYP0=; b=FF5R7xQ207cL+gu3jiofvDW2gKox8tuFjTtccuJWiYrD0rfS48mQQF2kjF5P66y9JB JUfWqiWhx37biaBeSTlORvH2dODk4ymaxT20FaQxoH/OUGUe7QsgWqINg4PI+Vqt6yte kjGy6M6JwuFtySHRZA0HwSXRKuNsHJOFHWvwXy2jAD0xf5WaemnxE0o3iBFSdWpYn0VT A5G7zaU1z63fJb89yY7axS69mG0xJyo0/psxjwUCn6SC0IhfWkR9IgPGzh79iTmbgxQ2 kD1NCur+8p1+tLsTf1a9C5Uoqi8RBo2OioJXx3jEdNX6xXNEsudzWSou6WML017fsYF5 v9HQ==
X-Gm-Message-State: AMCzsaVgNjL1YRfy9eixj5BQxsyK2/YkN+NlKZ+ko5Ic/AtH0AAgDuTc ob1kNx5SfrjfCwRMXtjbHIX8Jk4RDYrM5owh3g==
X-Google-Smtp-Source: AOwi7QDg/8Tn2NP1xxbEUJ9ORCeYfftQ4LGjmJI9/mv9xn0/YMiG5+dUcNGsMADqVrijEwfsSIE+3hT2SqCAcZzBpNA=
X-Received: by 10.157.86.186 with SMTP id o55mr3964657oth.88.1506950657930; Mon, 02 Oct 2017 06:24:17 -0700 (PDT)
MIME-Version: 1.0
Sender: thorgrin@gmail.com
Received: by 10.58.38.1 with HTTP; Mon, 2 Oct 2017 06:24:17 -0700 (PDT)
In-Reply-To: <b4c2a214-5e6c-861a-a394-ad26ae42bd21@cisco.com>
References: <CALbOe5M7QzMAmvd_mjoa_cqqbAd0idjiXpGbTBp3rtr-McXJNw@mail.gmail.com> <09DB7C34-E8A0-470D-A808-6875AB94FA9A@trammell.ch> <CALbOe5PArCC8XMgP6VajWRj=q6U4q9dLi8wqrqWOeGa30Mfyog@mail.gmail.com> <b4c2a214-5e6c-861a-a394-ad26ae42bd21@cisco.com>
From: Petr Velan <petr.velan@cesnet.cz>
Date: Mon, 02 Oct 2017 15:24:17 +0200
X-Google-Sender-Auth: J1yX96Iz_sjE5H6pY9y82TEGtJ8
Message-ID: <CALbOe5M+YzXN8VPKBU_04ZGxk43bauyfbZspiRUcFAjXeO4JOA@mail.gmail.com>
To: Benoit Claise <bclaise@cisco.com>
Cc: Petr Velan <petr.velan@cesnet.cz>, "Brian Trammell (IETF)" <ietf@trammell.ch>, ipfix@ietf.org
Content-Type: multipart/alternative; boundary="94eb2c0e71fca8fb96055a904a33"
Archived-At: <https://mailarchive.ietf.org/arch/msg/ipfix/t5eJqZVZvUwpEwvm_rFhChpCA2M>
Subject: Re: [IPFIX] templateLifePacket on IPFIX collector
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: IPFIX WG discussion list <ipfix.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ipfix>, <mailto:ipfix-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ipfix/>
List-Post: <mailto:ipfix@ietf.org>
List-Help: <mailto:ipfix-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ipfix>, <mailto:ipfix-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 02 Oct 2017 13:24:22 -0000

Hi Benoit,

I think that there is a slight confusion. I'm talking specifically about
__Collection Process__, not __Exporting Process__. My focus is on
templateLifeTime, optionsTemplateLifeTime, templateLifePacket, and
optionsTemplateLifePacket options and not at all about
templateRefreshTimeout, optionsTemplateRefreshTimeout,
templateRefreshPacket, and optionsTemplateRefreshPacket options.

With that cleared out, let my try to clarify:

On Mon, Oct 2, 2017 at 2:11 PM, Benoit Claise <bclaise@cisco.com> wrote:

> Hi Petr,
>
> Hi Brian,
>
> I understand that the Exporter is required to export the templates at
> regular intervals. It is not clear, whether "regular intervals" mean "time
> intervals" (as suggested by the templateRefreshTimeout example) or if it
> can mean "packet intervals" as well (as suggested by the RFC 6728).
>
> As mentioned, the two options come from NetFlow v9, with UDP.
>
> Moreover, this discusses only the Exporter Processes. My question was
> primarily about the Collecting Process, where the RFC 7011 specifies the
> following:
>
>    In order to minimize resource requirements for Templates that are no
>    longer being used by the Exporting Process, the Collecting Process
>    MAY associate a lifetime with each Template received in a Transport
>    Session.  Templates not refreshed by the Exporting Process within the
>    lifetime can then be discarded by the Collecting Process.  The
>    Template lifetime at the Collecting Process MAY be exposed by a
>    configuration parameter or MAY be derived from observation of the
>    interval of periodic Template retransmissions from the Exporting
>    Process.  In this latter case, the Template lifetime SHOULD default
>    to at least 3 times the observed retransmission rate.
>
> So the IPFIX configuration model only suggests that the Collecting Process
> can use the *TemplateLifePacket options without actually requiring their
> implementation anywhere. Is that correct?
>
> Not sure I understand.
> From RFC7011
>
>    Since UDP provides no method for reliable transmission of Templates,
>    Exporting Processes using UDP as the transport protocol MUST
>    periodically retransmit each active Template at regular intervals.
>    The Template retransmission interval MUST be configurable via, for
>    example, the templateRefreshTimeout and optionsTemplateRefreshTimeout
>    parameters as defined in [RFC6728 <https://tools.ietf.org/html/rfc6728>].  Default settings for these
>    values are deployment- and application-specific.
>
>
> If the collector is the one that configured the templateRefreshTimeout and
> optionsTemplateRefreshTimeout for the Exporter, then it knows this
> information. Alternatively, it can try to deduce it.
>
> This says that an __Exporting Process__ must implement some kind of the
template refresh timeout for UDP and that it must be configurable. However,
template lifetime tracking is only suggested for __Collecting Processes__
for the UDP protocol in the RFC 7011 and the template lifepacket expiration
is not mentioned at all. My original question was whether the template
lifepacket tracking was required/suggested by the IPFIX protocol as it was
present in the IPFIX information model. The answer to that question seems
to be _no_.


> Now, an editorial errata (more for completeness that anything else) could
> be that RFC7011 should mention RFC 6728 templateRefreshPacket and
> optionsTemplateRefreshPacket.
> However, it says "for example, the templateRefreshTimeout and
> optionsTemplateRefreshTimeout parameters as defined in [RFC6728
> <https://tools.ietf.org/html/rfc6728>]."
> So there might be other examples.
>
>
> On a related note, while reading through the RFC 6728, I've noticed that
> it still references the obsoleted RFC 5101, which is not good, especially
> regarding the template timeout on collecting process. The problem is that
> it forces the configuration of templateLifeTime and optionsTemplateLifeTime
> options per RFC 5101, Section 10.3.7 where it states:
>
>    The Collecting Process MUST associate a lifetime with each Template
>    (or another definition of an identifier considered unique within the
>    Transport Session) received via UDP.  Templates (and similar
>    definitions) not refreshed by the Exporting Process within the
>    lifetime are expired at the Collecting Process.
>
> However, the RFC 7011 obsoletes this (see my first quote) and changes this
> behaviour to optional. Would it make sense to update the RFC 6728 to
> reflect this change?
>
> Maybe.
>

That is a significant change from RFC 5101 which _required_ the template
lifetime tracking. The relevant text in RFC 6728 stil references RFC 5101
and suggest that implementing the template lifetime expiration for UDP it
is still required for __Collecting Processes_ by the IPFIX protocol, which
is no longer true.

The template expiration based on number of packets at the __Collecting
Process__ was not present anywhere in the RFC 3954. Therefore, I'm
surprised to find it in the IPFIX information model when it is not
mentioned in the standard. I fully agree that the templateRefreshPacket
makes complete sense at the __Exporting Process__, but my focus here is on
the __Collecting Process__.

Cheers,
Petr

>
> Regards, Benoit
>
>
> Cheers,
> Petr
>
> On Mon, Oct 2, 2017 at 10:05 AM, Brian Trammell (IETF) <ietf@trammell.ch>
> wrote:
>
>> hi Petr,
>>
>> This is specified in section 8.4 of RFC 7011 for UDP:
>>
>>    Since UDP provides no method for reliable transmission of Templates,
>>    Exporting Processes using UDP as the transport protocol MUST
>>    periodically retransmit each active Template at regular intervals.
>>    The Template retransmission interval MUST be configurable via, for
>>    example, the templateRefreshTimeout and optionsTemplateRefreshTimeout
>>    parameters as defined in [RFC6728].  Default settings for these
>>    values are deployment- and application-specific.
>>
>> For transports with reliable template lifetimes (TCP, SCTP), templates
>> require no refresh, since template state remains synchronized between
>> exporter and collector.
>>
>> Cheers,
>>
>> Brian
>>
>>
>> > On 2 Oct 2017, at 09:37, Petr Velan <petr.velan@cesnet.cz> wrote:
>> >
>> > Hello IPFIX masters,
>> >
>> > we are implementing an IPFIX collector and a question came up whether
>> to support templateLifePacket configuration as specified in RFC 6728 (
>> https://tools.ietf.org/html/rfc6728#section-4.5.2). It seems, that the
>> *LifePacket options are not mentioned anywhere else in the IPFIX related
>> documents and that the IPFIX protocol specifies only time based timeout
>> expiration. Therefore, my question is why the *LifePacket options are even
>> present in the RFC 6728 and whether it is necessary to implement them for
>> the collector (and exporter as well) to be compliant with the IPFIX
>> standard.
>> >
>> > I think that the whole *LifePacket option came from NetFlow v9 where
>> the informational RFC 3954 states that:
>> >       On a regular basis, the Exporter MUST send all the Template
>> >       Records and Options Template Records to refresh the Collector.
>> >       Template IDs have a limited lifetime at the Collector and MUST be
>> >       periodically refreshed.  Two approaches are taken to make sure
>> >       that Templates get refreshed at the Collector:
>> >             * Every N number of Export Packets.
>> >             * On a time basis, so every N number of minutes.
>> >       Both options MUST be configurable by the user on the Exporter.
>> >       When one of these expiry conditions is met, the Exporter MUST send
>> >       the Template FlowSet and Options Template.
>> >
>> >
>> > If that is the case, should the IPFIX standard specify something
>> similar as well? Otherwise, it would seem that removing the *LifePacket
>> options from the RFC 6728 would prevent further confusion.
>> >
>> > Kind regards,
>> > Petr Velan
>> >
>> > _______________________________________________
>> > IPFIX mailing list
>> > IPFIX@ietf.org
>> > https://www.ietf.org/mailman/listinfo/ipfix
>>
>>
>
>
> _______________________________________________
> IPFIX mailing listIPFIX@ietf.orghttps://www.ietf.org/mailman/listinfo/ipfix
>
>
>