Re: [OPSAWG] draft-gray-sampled-streaming-01.txt

"Gray, Andrew A" <Andrew.Gray@charter.com> Fri, 27 September 2019 16:39 UTC

Return-Path: <Andrew.Gray@charter.com>
X-Original-To: opsawg@ietfa.amsl.com
Delivered-To: opsawg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E88EE120A17 for <opsawg@ietfa.amsl.com>; Fri, 27 Sep 2019 09:39:13 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.598
X-Spam-Level:
X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VVdnEFQM-VoX for <opsawg@ietfa.amsl.com>; Fri, 27 Sep 2019 09:39:09 -0700 (PDT)
Received: from mail.chartercom.com (nce.mail.chartercom.com [142.136.234.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id ED4D5120A2E for <opsawg@ietf.org>; Fri, 27 Sep 2019 09:39:08 -0700 (PDT)
IronPort-SDR: obh6r74HzpKNr1FJbM+YYoS3P3ByQphNJltxifwotnLbJ1opLt+6twf5tO/FNJ0X2ZajOs5m3d hPUvJLDbfFHQ==
X-IronPort-AV: E=Sophos; i="5.64,556,1559538000"; d="scan'208,217"; a="69857565"
Received: from unknown (HELO NCEMEXGP011.CORP.CHARTERCOM.com) ([142.136.234.156]) by mail.chartercom.com with ESMTP; 27 Sep 2019 11:37:07 -0500
Received: from NCEMEXGP011.CORP.CHARTERCOM.COM (2002:8e88:ea10::8e88:ea10) by NCEMEXGP011.CORP.CHARTERCOM.com (2002:8e88:ea10::8e88:ea10) with Microsoft SMTP Server (TLS) id 15.0.1365.1; Fri, 27 Sep 2019 11:37:07 -0500
Received: from NCEMEXGP011.CORP.CHARTERCOM.COM ([fe80::e832:53f4:2a29:ec5f]) by NCEMEXGP011.CORP.CHARTERCOM.com ([fe80::e832:53f4:2a29:ec5f%19]) with mapi id 15.00.1365.000; Fri, 27 Sep 2019 11:37:06 -0500
From: "Gray, Andrew A" <Andrew.Gray@charter.com>
To: Dmytro Shytyi <ietf.dmytro@shytyi.net>
CC: opsawg <opsawg@ietf.org>
Thread-Topic: [OPSAWG] draft-gray-sampled-streaming-01.txt
Thread-Index: AQHVdVHMa2Gu6YJV0kWIbkLyuHaVvw==
Date: Fri, 27 Sep 2019 16:37:06 +0000
Message-ID: <3C0BB128-58D3-472F-91DA-26780C3A7C53@charter.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [142.136.234.123]
Content-Type: multipart/alternative; boundary="_000_3C0BB12858D3472F91DA26780C3A7C53chartercom_"
MIME-Version: 1.0
Archived-At: <https://mailarchive.ietf.org/arch/msg/opsawg/uUCSsGT0pgqTzoJfFofojtOMDNw>
Subject: Re: [OPSAWG] draft-gray-sampled-streaming-01.txt
X-BeenThere: opsawg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: OPSA Working Group Mail List <opsawg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/opsawg>, <mailto:opsawg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/opsawg/>
List-Post: <mailto:opsawg@ietf.org>
List-Help: <mailto:opsawg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/opsawg>, <mailto:opsawg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 27 Sep 2019 16:39:22 -0000

That’s a good point, and one I hadn’t considered before.  I think an implementation would be relatively straight forward for this use case – your capture points would be listed as one per interface, and the points reporting a 1:1 mapping of the interfaces.

I believe the YANG model is flexible enough to describe the PCAP-NG enhanced packet block format with its current definitions, minus having the extra options and block total length after the variable length packet data – right now the model only allows one variable length field (which would be the packet data).
The capture filters would be relatively straight-forward to convert to the command line arguments, as you mention, so the filtering support would be easy as well.

This may actually be useful to write up as a proof-of-concept/running-code type example for the draft.  I would need to ponder this a little bit more and give it a try and see how it would work against the prototype receiver I wrote for lab/testing work.

From: Dmytro Shytyi <ietf.dmytro@shytyi.net>
Date: Friday, September 27, 2019 at 12:50 AM
To: "Gray, Andrew A" <Andrew.Gray@charter.com>
Cc: opsawg <opsawg@ietf.org>
Subject: Re: [OPSAWG] draft-gray-sampled-streaming-01.txt

Hello,
Is it possible that this approach is applicable to non ASIC equipment?
For example the devices that are running NFVIs (uCPE I-D "opsawg vysm") with x86 capabilities. Normally the tcpdump/tshark can be started in this kind of devices. One can imagine that this yang model variables could be mapped to the filter arguments  on the sniffing software?
Best,
Dmytro Shytyi.
Get BlueMail for Android<http://www.bluemail.me/r?b=15656>
On Sep 26, 2019, at 22:09, "Gray, Andrew A" <andrew.gray@charter.com<mailto:andrew.gray@charter.com>> wrote:

I have updated draft-gray-sampled-streaming with the feedback I have received to do (thank you to those that reviewed and provided that feedback).

-01 includes additional clarifications about format, clarifying how traffic direction is handled, and some other cleanups.  I have not yet had a chance to look at Flowspec for rule definitions, so that part is not in yet.

Please feel free to take a look, and am interested in getting feedback back.  Thanks!

A new version of I-D, draft-gray-sampled-streaming-01.txt
has been successfully submitted by Andrew Gray and posted to the
IETF repository.

Name:  draft-gray-sampled-streaming
Revision: 01
Title:  Sampled Traffic Streaming
Document date: 2019-09-25
Group:  Individual Submission
Pages:  18
URL:            https://www.ietf.org/internet-drafts/draft-gray-sampled-streaming-01.txt
Status:         https://datatracker.ietf.org/doc/draft-gray-sampled-streaming/
Htmlized:       https://tools.ietf.org/html/draft-gray-sampled-streaming-01
Htmlized:       https://datatracker.ietf.org/doc/html/draft-gray-sampled-streaming
Diff:           https://www.ietf.org/rfcdiff?url2=draft-gray-sampled-streaming-01

Abstract:
   This document standardizes the means of requesting a sampled capture
   stream from a router, receiving details about the resulting data
   flow, and the structure of the data flow itself.  This is
   specifically tailored to having various hardware ASICs be able to
   perform this operation as quickly as possible, by allowing
   communication of the specific bit formats of headers applied to the
   packet flow, in a way that enhances interoperability between sources
   and sinks.  Historically, NetFlow and its ilk have been used for
   these use cases, however the growth in hardware forward speeds is far
   outpacing the growth in CPU speeds, and the CPU-heavy parts of
   NetFlow is resulting in a reduction of sampling rates that include
   all of the fields provided by NetFlow that require CPU lookups.



E-MAIL CONFIDENTIALITY NOTICE:
The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

________________________________

OPSAWG mailing list
OPSAWG@ietf.org
https://www.ietf.org/mailman/listinfo/opsawg
E-MAIL CONFIDENTIALITY NOTICE: 
The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.