Re: [netconf] restconf collections

Kent Watsen <kent+ietf@watsen.net> Fri, 02 October 2020 14:46 UTC

Return-Path: <01000174e9c75096-5f451248-48e7-4add-a56b-9789cddd3e56-000000@amazonses.watsen.net>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 11EA93A10EB for <netconf@ietfa.amsl.com>; Fri, 2 Oct 2020 07:46:30 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.895
X-Spam-Level:
X-Spam-Status: No, score=-1.895 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=amazonses.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R313IxK8lyZo for <netconf@ietfa.amsl.com>; Fri, 2 Oct 2020 07:46:29 -0700 (PDT)
Received: from a8-96.smtp-out.amazonses.com (a8-96.smtp-out.amazonses.com [54.240.8.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2CA9D3A1092 for <netconf@ietf.org>; Fri, 2 Oct 2020 07:46:29 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=224i4yxa5dv7c2xz3womw6peuasteono; d=amazonses.com; t=1601649988; h=From:Message-Id:Content-Type:Mime-Version:Subject:Date:In-Reply-To:Cc:To:References:Feedback-ID; bh=WbpyfFfD+yI4N57PxB1CCLsx66+hD+YZ9ta06+JuHhU=; b=VeZmLhkIzaWxuToscBDm69eomxrKj9mvPRp5PzmElV0sGRb4wtL0nYCaBolvzXG8 Kjd0LwXLwJBLDTAimtXfc8vtGW3qDSYfR0mT/WRPvDWgoZTGpA/zzNp6NPojZVALzc6 u+xUMFCTq9lBUCepQAf8byAdYsqnRUh31a8gQhi0=
From: Kent Watsen <kent+ietf@watsen.net>
Message-ID: <01000174e9c75096-5f451248-48e7-4add-a56b-9789cddd3e56-000000@email.amazonses.com>
Content-Type: multipart/alternative; boundary="Apple-Mail=_C6CE3C3F-81F0-48B6-9854-D005124FEDBC"
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\))
Date: Fri, 2 Oct 2020 14:46:27 +0000
In-Reply-To: <CAL73O_x_mPwB6S-=UdASFK_H9f7CkeYU270ZZHX5xf9eLM=iAQ@mail.gmail.com>
Cc: =?utf-8?Q?Martin_Bj=C3=B6rklund?= <mbj+ietf@4668.se>, "netconf@ietf.org" <netconf@ietf.org>
To: Hongwei Li <flycoolman@gmail.com>
References: <01000174e4610ae6-d1488376-ae7e-4d95-ba7e-005fc44b9592-000000@email.amazonses.com> <20201001.165548.1275939966226069939.id@4668.se> <01000174e5ba8cc5-39edb446-48d9-4896-a6b5-dcf80f962762-000000@email.amazonses.com> <20201002.082331.2242297379846191383.id@4668.se> <01000174e96cd06f-4ecb06ac-c7ac-4233-bd5d-97d3dbc74830-000000@email.amazonses.com> <CAL73O_x_mPwB6S-=UdASFK_H9f7CkeYU270ZZHX5xf9eLM=iAQ@mail.gmail.com>
X-Mailer: Apple Mail (2.3608.80.23.2.2)
X-SES-Outgoing: 2020.10.02-54.240.8.96
Feedback-ID: 1.us-east-1.DKmIRZFhhsBhtmFMNikgwZUWVrODEw9qVcPhqJEI2DA=:AmazonSES
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/CMJrh9H1jQIEZe-vLoEHPe-w8DU>
Subject: Re: [netconf] restconf collections
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 02 Oct 2020 14:46:30 -0000

Hi Hongwei,

> [HL] A user case for time series data (e.g., logs), customers want to get the data in between Time A and Time B. Do we use timestamp filter here?


Some events have only one timestamp.  The database likely has a primary key (e.g., "record-id”) and a separate “timestamp” field.  For all intents and purposes, the logs are persisted in time-order, so no additional sorting is needed.  The physical order is good enough.

However, other events may have distinct "time-generated" and "time-received” fields.  This is most notable for a “log-receiver” (or just “receiver", in rfc8639 parlance), as there may be a delay between when a log is generated by a publisher and when it is persisted by the receiver.   In this case, the user-expectation is undoubtedly to sort on “time-generated”. The “record-id” and “time-received” fields are physically in order (same as above), but “time-generated” could be all over the place, so a sort is needed.

FWIW, if the goal is to find logs generated in a window of time around a timestamp, a fast-filter can be used to whittle down the result-set to a (hopefully) manageable size before the slow-sort as follows:

	filter
		"time-generated >= timestamp-of-interest - some-window”
		and
		“time-received <= timestamp-of-interest + some-window"
	sort-by
		“time-generated"


K.