Re: [netconf] restconf collections

Hongwei Li <flycoolman@gmail.com> Fri, 02 October 2020 15:09 UTC

Return-Path: <flycoolman@gmail.com>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 776AC3A1654 for <netconf@ietfa.amsl.com>; Fri, 2 Oct 2020 08:09:21 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Level:
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id u51LmrMtZDVo for <netconf@ietfa.amsl.com>; Fri, 2 Oct 2020 08:09:19 -0700 (PDT)
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com [IPv6:2a00:1450:4864:20::12d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4C4993A1651 for <netconf@ietf.org>; Fri, 2 Oct 2020 08:09:09 -0700 (PDT)
Received: by mail-lf1-x12d.google.com with SMTP id b22so2241410lfs.13 for <netconf@ietf.org>; Fri, 02 Oct 2020 08:09:09 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=rJSLI5yI5rQ0Zs2OBmCiHeJuFQ2lDnY5UflA9KDYFlw=; b=l3fTYLnGjxWBC2OvHgdZGWPKUeSxak7jkoTZcnGRuW9CIIzpXRq2Konm48hvd0otnw b819yJoIHWHEX2LA8YJPyc4BffiR+rl7acBZPTk9DdW6qcmnqpISJZpDv5FwEfqAFry0 Hf+6rgwEqX4BRO46ILDycEQuxKOBCjsgpG3wpby2qJj7H61vy7X82y7D1OCrDeKP86Da rAAr0SAk75A8xLCF2WV1ffoK17aFH4GSsu4PAYTJSiXqZFQI6rAtgnCUNXHMzQkwI3Vi PGvCuPL9Op5sp1WkqwiL8rm79Q9ys9xmKubxlpBxXotkJZMpop5zG4zNjEaAQu3yCkyd 4PCQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=rJSLI5yI5rQ0Zs2OBmCiHeJuFQ2lDnY5UflA9KDYFlw=; b=QyXJWkaC5S4sr+RgG2ZP0263XQrsgKMFmB2fx5yRkH9YD71Q7HLAOdy5LZ1hpQNiBB DaYnPFYCZg4DFDbu6WynMaS/rGIrqc3qpE8JPANKc1GMJFCxpF4uSpWqD9+pYe/Nu3Hn qxPlhOiFeS/hTsZr1he6TOodJlwlmJ+d1qp7XQcJFnzylcPmfrrOuvYePEFOgrPpMF82 s0Gm0Pig0KuKiQk5DO1y2giZ6EIZIWHJkRf2dFnawlkK57xYwoh5MYGcGGBqGHBvMM2R WX1a52CtKJwKqG3VqWscNi7/Yc7Gu5ZuqVIQcMSpoUBes+41TH3dwZ5rDUNCc2j65GCI HpKQ==
X-Gm-Message-State: AOAM5300ed4qjrYOo/yyWJ8eLyaCW3jwIO78aXjrKh+VXsLpb1gh8LKx eeEEQgF4XUHoWCBIaCu3574GI5O+t0EWPyIoybMg+6/bBTFWeg==
X-Google-Smtp-Source: ABdhPJySVo8Pm5DwggRJ250EanxoY0MxOvr5QqrF/YEMn0VJtMuYNgoFtWDKvz9x+LrSj2dXB//J6g4tAg1QKwf9Db8=
X-Received: by 2002:a19:5f5d:: with SMTP id a29mr1138089lfj.259.1601651347482; Fri, 02 Oct 2020 08:09:07 -0700 (PDT)
MIME-Version: 1.0
References: <01000174e4610ae6-d1488376-ae7e-4d95-ba7e-005fc44b9592-000000@email.amazonses.com> <20201001.165548.1275939966226069939.id@4668.se> <01000174e5ba8cc5-39edb446-48d9-4896-a6b5-dcf80f962762-000000@email.amazonses.com> <20201002.082331.2242297379846191383.id@4668.se> <01000174e96cd06f-4ecb06ac-c7ac-4233-bd5d-97d3dbc74830-000000@email.amazonses.com> <CAL73O_x_mPwB6S-=UdASFK_H9f7CkeYU270ZZHX5xf9eLM=iAQ@mail.gmail.com> <01000174e9c75096-5f451248-48e7-4add-a56b-9789cddd3e56-000000@email.amazonses.com>
In-Reply-To: <01000174e9c75096-5f451248-48e7-4add-a56b-9789cddd3e56-000000@email.amazonses.com>
From: Hongwei Li <flycoolman@gmail.com>
Date: Fri, 2 Oct 2020 11:08:56 -0400
Message-ID: <CAL73O_z+zy9wN9YOoGuA1VOTKxj4FL2Jj0N+cQ8fScT04J8-UA@mail.gmail.com>
To: Kent Watsen <kent+ietf@watsen.net>
Cc: =?UTF-8?Q?Martin_Bj=C3=B6rklund?= <mbj+ietf@4668.se>, "netconf@ietf.org" <netconf@ietf.org>
Content-Type: multipart/alternative; boundary="0000000000009eeb5805b0b18336"
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/ua1PUZwylQ_1MsSXQ3hnmyJ3kE4>
Subject: Re: [netconf] restconf collections
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 02 Oct 2020 15:09:22 -0000

Thanks Kent
This is clear enough. Just want to make sure we cover that.
The next question is: can we avoid the slow-sort? Like to split the request
automatically or something else? IMO, this is more on implementation side.

On Fri, Oct 2, 2020 at 10:46 AM Kent Watsen <kent+ietf@watsen.net> wrote:

> Hi Hongwei,
>
> [HL] A user case for time series data (e.g., logs), customers want to get
>> the data in between Time A and Time B. Do we use timestamp filter here?
>>
>
> Some events have only one timestamp.  The database likely has a primary
> key (e.g., "record-id”) and a separate “timestamp” field.  For all intents
> and purposes, the logs are persisted in time-order, so no additional
> sorting is needed.  The physical order is good enough.
>
> However, other events may have distinct "time-generated" and
> "time-received” fields.  This is most notable for a “log-receiver” (or just
> “receiver", in rfc8639 parlance), as there may be a delay between when a
> log is generated by a publisher and when it is persisted by the receiver.   In
> this case, the user-expectation is undoubtedly to sort on “time-generated”. The
> “record-id” and “time-received” fields are physically in order (same as
> above), but “time-generated” could be all over the place, so a sort is
> needed.
>
> FWIW, if the goal is to find logs generated in a window of time around a
> timestamp, a fast-filter can be used to whittle down the result-set to a
> (hopefully) manageable size before the slow-sort as follows:
>
> filter
> "time-generated >= timestamp-of-interest - some-window”
> and
> “time-received <= timestamp-of-interest + some-window"
> sort-by
> “time-generated"
>
>
> K.
>
>