[bmwg] Pasted review answers - RE: WGLC: draft-ietf-bmwg-ipflow-meth-04
"Jan Novak (janovak)" <janovak@cisco.com> Sun, 02 October 2011 18:20 UTC
Return-Path: <janovak@cisco.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 32EA421F85EF for <bmwg@ietfa.amsl.com>; Sun, 2 Oct 2011 11:20:26 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.174
X-Spam-Level:
X-Spam-Status: No, score=-10.174 tagged_above=-999 required=5 tests=[AWL=0.424, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bj+ilXMOpJ1E for <bmwg@ietfa.amsl.com>; Sun, 2 Oct 2011 11:20:19 -0700 (PDT)
Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id D1F5C21F85DB for <bmwg@ietf.org>; Sun, 2 Oct 2011 11:20:15 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=janovak@cisco.com; l=89263; q=dns/txt; s=iport; t=1317579796; x=1318789396; h=mime-version:subject:date:message-id:in-reply-to: references:from:to:cc; bh=LPV60rJbIn+9T/sZrMk8vu0UIIZXP4VfX+BIdRiJOug=; b=R2LRonKzfVmReURwmnM4fe6w3qqTCQvoD7PeB9iJYUv7Qa0z30aYC7Cy 14Z9PsuYW3ojflWajtfMgnxIATYXimwbrbydn7Kd/5Gb2DKb4V2pEzvyn bleYa4nWUWm1BpslxG/bj3uh+XIbZJJ+1MrEdoy5e3JlHrUWlV4e746rC k=;
X-IronPort-AV: E=Sophos; i="4.68,477,1312156800"; d="scan'208,217"; a="56928167"
Received: from ams-core-2.cisco.com ([144.254.72.75]) by ams-iport-2.cisco.com with ESMTP; 02 Oct 2011 18:23:15 +0000
Received: from xbh-ams-101.cisco.com (xbh-ams-101.cisco.com [144.254.74.71]) by ams-core-2.cisco.com (8.14.3/8.14.3) with ESMTP id p92INFsd015031; Sun, 2 Oct 2011 18:23:15 GMT
Received: from xmb-ams-212.cisco.com ([144.254.75.23]) by xbh-ams-101.cisco.com with Microsoft SMTPSVC(6.0.3790.4675); Sun, 2 Oct 2011 20:23:15 +0200
X-MimeOLE: Produced By Microsoft Exchange V6.5
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01CC8130.59AAF7C7"
Date: Sun, 02 Oct 2011 20:23:14 +0200
Message-ID: <C95CC96B171AF24CA1BB6CA3C52D0BA0010DB49C@XMB-AMS-212.cisco.com>
In-Reply-To: <4E6A0E03.3060807@cisco.com>
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Thread-Topic: Pasted review answers - RE: WGLC: draft-ietf-bmwg-ipflow-meth-04
Thread-Index: Acxu8IO6DLQZWOgkTCWugo/pPFbxFASP6S+g
References: <4DF60A70.4070902@cisco.com> <4DFF103A.1000209@cisco.com> <C95CC96B171AF24CA1BB6CA3C52D0BA0A3F270@XMB-AMS-212.cisco.com> <4E6A0E03.3060807@cisco.com>
From: "Jan Novak (janovak)" <janovak@cisco.com>
To: "Paul Aitken (paitken)" <paitken@cisco.com>
X-OriginalArrivalTime: 02 Oct 2011 18:23:15.0616 (UTC) FILETIME=[5A0DAA00:01CC8130]
X-Mailman-Approved-At: Sun, 02 Oct 2011 19:29:51 -0700
Cc: Al Morton <acmorton@att.com>, bmwg@ietf.org
Subject: [bmwg] Pasted review answers - RE: WGLC: draft-ietf-bmwg-ipflow-meth-04
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 02 Oct 2011 18:20:26 -0000
Just pasted answers to the draft-03 review: http://www.ietf.org/mail-archive/web/bmwg/current/msg02413.html ==========PJ =============== OK, I'm happy with that. Please clarify this in section 4.3.3, eg: Various Flow monitoring implementations might use different default values regarding the export of Control Information [RFC5470]. Note that Control Information includes IPFIX Options and Templates [RFC5101]. The Flow Export corresponding to Control Information ... ==========PJ =============== Clarified - see the text please ==========PJ =============== Note that Control Information includes IPFIX Options and Templates [RFC5101]. ==========PJ =============== Yes, I know that - what is your point here please ?? ==========PJ =============== Per Brian's feedback, please add to section 4.5: Packet sampling and flow sampling is out of scope of this document. This document applies to situations without packet or flow sampling. ==========PJ =============== Added ==========PJ =============== Section 1: remove "the": is provided in the section 3.3. (And many more times, throughout the document - eg, at the end of Section 3.1, in Section 3.4.1, at the very end of 3.4.2, ... Please search yourself.) ==========PJ =============== Searched and removed or occurences. ==========PJ =============== In section 1, at the bottom of page 2, write "DUT's" and remove "support": The only restriction may be the DUT's lack of support for Flow monitoring support of the particular traffic type. ==========PJ =============== Changed ==========PJ =============== Section 2.2.5: mention that there SHOULD NOT be any export filtering, so that all the expired cache entries are exported. If there is export filtering and it can't be disabled, this needs to be noted. ==========PJ =============== Added as suggested ==========PJ =============== Also in section 3.1, I had said: ==========PJ =============== I answered this in the previous review already: http://www.ietf.org/mail-archive/web/bmwg/current/msg02413.html ==========PJ =============== Section 3.4.1: In this scenario every packet seen by the DUT creates a new Cache entry and forces the DUT to the full Cache processing fill the Cache instead of just updating packet and byte counters on of an already existing Cache entry. ==========PJ =============== Changed ==========PJ =============== Section 3.4.2: the indentation of the second line in each point is wrong - compare with other a/b/c immediately above and below: ==========PJ =============== Changed ==========PJ =============== Section 4.1: s/if/whether/ The ideal way to implement the measurement is by using a single device to provide the sender and receiver capabilities with a sending port and a receiving port. This allows for an easy check if whether all the traffic sent by the sender was re-transmitted by the DUT and received at the receiver. PJ: If the sender and receiver are independent (ie, two devices), then there needs to be another link directly from the sender to the receiver (ie, not through the DUT), so the receiver can make that comparison. - how can the comparison be made if the tester is unable to implement your suggestion of a single device? ==========PJ =============== Changed. In our tests we don't have one device but we still know what was sent by configuration of the sender and receiver. It is not ideal as stated in the draft but still not very difficult to figure out. I don't see any importance/relevance of this - it is a standard function of any commercial traffic generator. ==========PJ =============== Section 4.1: In all measurements, the export interface MUST have enough bandwidth to transmit Flow Export data without congestion. In other words, the export interface MUST NOT be a bottleneck during the measurement. PJ: What if the receiving interface, or the collector itself, is a bottleneck? Previously you said it doesn't matter, and I disagreed. - data could be lost at the receiving end, making the DUT look worse than it really is. ==========PJ =============== I answered this in the previous review already: http://www.ietf.org/mail-archive/web/bmwg/current/msg02413.html I believe this is all covered in the section 4.4 already ==========PJ =============== Section 4.2: The DUT export interface (see Figure 2) MUST be configured with sufficient output buffers to avoid dropping the Flow Export data due to a simple lack of resources in the interface hardware. PJ: then it might not be configured as it would in real life conditions. Wouldn't it be better to record the real-life results than to tweak the settings just for testing purposes? - eg, consider if the DUT operates with insufficient buffers in the real-world scenario, and therefore drops a lot more data than when testing under "ideal" conditions in the lab. ==========PJ =============== I would like to disagree here. We are not measuring real life scenario here. We are trying to measure the maximum what the DUT can do. In real life you would also configure the export interface (if there is one dedicated at all - see the discussion in Appendix B) not to drop the export packets. ==========PJ =============== Section 4.3: The DUT SHOULD support IPFIX [RFC5101] Say why? ==========PJ =============== This has been changed several times as the result of the previous reviews. I don't know what text you would like there. Please suggest something. The previous text was "The DUT SHOULD support IPFIX [RFC5101] to allow easier results comparison" and your/someone else's question was "What difference it would make ??" - sorry no idea what you want from me here, it just feels like common sense to have same protocol for testing everywhere if possible ?? ==========PJ =============== Section 4.3.1: In the case when both ingress and egress Flow monitoring is enabled on one DUT the results analysis needs to take into account that each Flow will be represented in the DUT Cache by two Flow Records (one for each direction) and therefore also the Flow Export will contain those two Flow Records. PJ: you assume a single cache. ingress and egress may potentially be recorded in separate caches. This may, or may not, impact the performance. - so mention that whether the combined ingress and egress traffic is measured in one cache or each separately in its own cache, may impact performance and should be recorded. ==========PJ =============== I have added text in 4.3.2.l If it is not sufficient please suggest something more appropriate. ==========PJ =============== Section 4.3.2: add "of", remove "the", and move "instantly" : The Cache's Inactive and Active Timeouts MUST be known and taken into account when designing the measurement as specified in section 5. If the Flow monitoring implementation allows only timeouts of zero (e.g. immediate timeout or non-existent Cache) then the measurement conditions in the section 5 are fulfilled inherently without any additional configuration. The DUT simply instantly exports instantly information about every single packet. PJ: Assuming that the cache works with timeouts. What if it uses some other mechanism, eg number of packets in the flow? - state what impact other such mechanisms may have. ==========PJ =============== Changed the text. I don't know about any such impact. Could you elaborate please ?? ==========PJ =============== Section 4.3.3: add "of" : Section 10 of [RFC5101] and section 8.1 of [RFC5470] discuss ==========PJ =============== Changed. ==========PJ =============== Section 4.3.3: add "of" : Section 10 of [RFC5101] and section 8.1 of [RFC5470] discuss ==========PJ =============== Added ==========PJ =============== Section 4.3.3: The Exporting Process SHOULD be configured with IPFIX [RFC5101] as the protocol to use to format the Flow Export data. If the Flow monitoring implementation does not support it, proprietary protocols MAY be used. PJ: what difference would this make to the testing? Eg, NFv5 is a proprietary protocol which has smaller data records and no templates - so it requires less export bandwidth. Wouldn't the results look better if NFv5 was used rather than IPFIX? - at least state that since proprietary protocols may make a considerable difference to the testing, the exact protocol being tested (and any related configuration parameters) MUST be recorded and only similar protocols should be compared. ==========PJ =============== This section has been edited many times to satisfy various reviewers ideas. I have tried another one. Of it does not suit please provide some text. This also answers your comment about 4.3. ==========PJ =============== Section 4.3.5, remove "on" : The test report should therefore contain information containing on how many Metering and Exporting processes were configured on the DUT for the selected Observation Points. ==========PJ =============== Removed ==========PJ =============== Section 4.3.6: insert "The" The forwarding performance document [RFC5695] specifies ==========PJ =============== ==========PJ =============== Section 4.4: However if the Collector is also used to decode the Flow Export data then it SHOULD support IPFIX [RFC5101] for easier results analysis. PJ: Again, why does IPFIX make it easier? ==========PJ =============== Please suggest some text, this has gone through many variations as well. It is answered in the discussion above. ==========PJ =============== Section 4.9.1: A packet with destination IP address equal to A is sent every 10 seconds, so the Cache entry would be refreshed in the Cache every 10 seconds. However, the Inactive Timeout is 5 seconds, so the Cache entries will expire from the Cache due to the Inactive Timeout and when a new packet is sent with the same IP address A it will create a new entry in the Cache. PJ: theoretically. In practice... the DUT has to check those 10,000 cache entries within the 10 seconds to ensure that expired cache entries are exported. If it checks 1,000 cache entries per second, it may only just be ready to expire the existing cache entry when the new packet arrives. Therefore the new packet may sometimes be added to the existing cache entry, giving occasional 2 packet flows! - so note that this behaviour depends upon the design an efficiency of the cache ager, and incidences of multi-packet flows observed during this test should be noted. ==========PJ =============== Added text as suggested. ==========PJ =============== At the end of section 4.9.1: with large Cache Sizes and high packet rate where the DUT's actual ==========PJ =============== Changed ==========PJ =============== Section 4.9.2: So each stream has a packet rate of 10 packets per second. The packets A packet with destination IP address equal to A is sent every 0.1 second, so it means that the Cache entry is refreshed in the Cache ==========PJ =============== Changed ==========PJ =============== I think you could work around all the config issues by simply saying that the DUT should conform to the IPFIX Config model described in draft-ietf-ipfix-configuration-model (which is in the RFC Editor's queue, waiting for the IPFIX MIB). ==========PJ =============== Tried to add some text to 5.1, if it is not suitable please provide some. ==========PJ =============== Section 5.3: the "Page 19" header/footer is broken - search for [Page 19]. ==========PJ =============== Couldn't see it broken, but tried ==========PJ =============== PJ: show that calculation here. - show the exact calculation you have in mind, to ensure that every reader calculates the same. ==========PJ =============== The formula is just above the text you point to, so it is trivial to figure out: The measurement time interval is then calculated as the difference (stop time) - (start time) - (Inactive Timeout). ==========PJ =============== PJ: I'm confused by the final part. What are you trying to avoid? ==========PJ =============== I have answered this in the previous review, Please read http://www.ietf.org/mail-archive/web/bmwg/current/msg02413.html to progress any further or simply suggest some text ==========PJ =============== Section 7: "The" shouldn't be capitalised: ==========PJ =============== Changed ==========PJ =============== B.1 add "the" and "a": ==========PJ =============== Added The climate of Edinburgh is such that the weak succumb young .... and the strong envy them. Dr. Johnson From: Paul Aitken (paitken) Sent: 09 September 2011 14:01 To: Jan Novak (janovak) Cc: Al Morton; bmwg@ietf.org Subject: Re: WGLC: draft-ietf-bmwg-ipflow-meth-03 Jan, I reviewed the changes from -02 to -03, then the changes from -01 to -03, and finally I reviewed what you've done in -03 in response to my previous feedback. The new version is a big improvement over -01 which I last reviewed - so I have many minor comments, along with several points you seem to have missed or overlooked. Please see my comments inline below. I look forward to seeing a -04. P.
- [bmwg] WGLC: draft-ietf-bmwg-ipflow-meth-00 Al Morton
- Re: [bmwg] [IPFIX] WGLC: draft-ietf-bmwg-ipflow-m… Peter Phaal
- Re: [bmwg] [IPFIX] WGLC: draft-ietf-bmwg-ipflow-m… Jan Novak (janovak)
- Re: [bmwg] WGLC: draft-ietf-bmwg-ipflow-meth-00 Al Morton
- Re: [bmwg] WGLC: draft-ietf-bmwg-ipflow-meth-00 Al Morton
- Re: [bmwg] WGLC: draft-ietf-bmwg-ipflow-meth-00 Jan Novak (janovak)
- Re: [bmwg] WGLC: draft-ietf-bmwg-ipflow-meth-00 Al Morton
- [bmwg] WGLC: draft-ietf-bmwg-ipflow-meth-00 Al Morton
- Re: [bmwg] WGLC: draft-ietf-bmwg-ipflow-meth-00 Paul Aitken
- Re: [bmwg] WGLC: draft-ietf-bmwg-ipflow-meth-00 Brian Trammell
- Re: [bmwg] WGLC: draft-ietf-bmwg-ipflow-meth-00 Paul Aitken
- Re: [bmwg] WGLC: draft-ietf-bmwg-ipflow-meth-00 -… Jan Novak (janovak)
- Re: [bmwg] WGLC: draft-ietf-bmwg-ipflow-meth-00 -… Brian Trammell
- Re: [bmwg] WGLC: draft-ietf-bmwg-ipflow-meth-00 -… Jan Novak (janovak)
- Re: [bmwg] WGLC: draft-ietf-bmwg-ipflow-meth-03 Paul Aitken
- Re: [bmwg] WGLC: draft-ietf-bmwg-ipflow-meth-00 -… Jan Novak (janovak)
- Re: [bmwg] WGLC: draft-ietf-bmwg-ipflow-meth-03 Jan Novak (janovak)
- Re: [bmwg] WGLC: draft-ietf-bmwg-ipflow-meth-04 Jan Novak (janovak)
- [bmwg] Pasted review answers - RE: WGLC: draft-ie… Jan Novak (janovak)
- [bmwg] Issue 1: Ref? to IPFIX Options and Templat… Al Morton
- [bmwg] Flow Export Issue#2: Test Device Capacity Al Morton
- [bmwg] Flow Export Issue #3: Al Morton
- [bmwg] Flow Export Issue#4: Al Morton
- [bmwg] Flow Export Issue #5: Metering Configurati… Al Morton
- Re: [bmwg] Flow Export Issue#2: Test Device Capac… Jan Novak (janovak)
- Re: [bmwg] Flow Export Issue #3: Jan Novak (janovak)
- Re: [bmwg] Flow Export Issue#4: Jan Novak (janovak)
- Re: [bmwg] Flow Export Issue #5: Metering Configu… Jan Novak (janovak)