Re: [netconf] restconf collections

Qin Wu <bill.wu@huawei.com> Fri, 02 October 2020 10:01 UTC

Return-Path: <bill.wu@huawei.com>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C2D5F3A0EF0 for <netconf@ietfa.amsl.com>; Fri, 2 Oct 2020 03:01:37 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RTGi7jLLLoHM for <netconf@ietfa.amsl.com>; Fri, 2 Oct 2020 03:01:35 -0700 (PDT)
Received: from huawei.com (lhrrgout.huawei.com [185.176.76.210]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CC0193A0F78 for <netconf@ietf.org>; Fri, 2 Oct 2020 03:01:34 -0700 (PDT)
Received: from lhreml714-chm.china.huawei.com (unknown [172.18.7.108]) by Forcepoint Email with ESMTP id 02931677BE54F8F9A033 for <netconf@ietf.org>; Fri, 2 Oct 2020 11:01:33 +0100 (IST)
Received: from lhreml714-chm.china.huawei.com (10.201.108.65) by lhreml714-chm.china.huawei.com (10.201.108.65) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1913.5; Fri, 2 Oct 2020 11:01:32 +0100
Received: from DGGEML405-HUB.china.huawei.com (10.3.17.49) by lhreml714-chm.china.huawei.com (10.201.108.65) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.1.1913.5 via Frontend Transport; Fri, 2 Oct 2020 11:01:32 +0100
Received: from DGGEML531-MBS.china.huawei.com ([169.254.5.245]) by dggeml405-hub.china.huawei.com ([10.3.17.49]) with mapi id 14.03.0487.000; Fri, 2 Oct 2020 18:01:28 +0800
From: Qin Wu <bill.wu@huawei.com>
To: Andy Bierman <andy@yumaworks.com>, =?utf-8?B?TWFydGluIEJqw7Zya2x1bmQ=?= <mbj+ietf@4668.se>
CC: Netconf <netconf@ietf.org>
Thread-Topic: [netconf] restconf collections
Thread-Index: AdaYobKg+bjeJP+oT1CByuaRwLyn8g==
Date: Fri, 2 Oct 2020 10:01:27 +0000
Message-ID: <B8F9A780D330094D99AF023C5877DABAADA3AA34@dggeml531-mbs.china.huawei.com>
Accept-Language: zh-CN, en-US
Content-Language: zh-CN
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [10.46.122.121]
Content-Type: multipart/alternative; boundary="_000_B8F9A780D330094D99AF023C5877DABAADA3AA34dggeml531mbschi_"
MIME-Version: 1.0
X-CFilter-Loop: Reflected
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/tBk701-KowOxmzSgGNxKTkm-Xzk>
Subject: Re: [netconf] restconf collections
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 02 Oct 2020 10:01:38 -0000

A few thought on negative offset.
发件人: netconf [mailto:netconf-bounces@ietf.org] 代表 Andy Bierman
发送时间: 2020年10月2日 0:17
收件人: Martin Björklund <mbj+ietf@4668.se>
抄送: Netconf <netconf@ietf.org>
主题: Re: [netconf] restconf collections



On Thu, Oct 1, 2020 at 7:57 AM Martin Björklund <mbj+ietf@4668.se<mailto:mbj%2Bietf@4668.se>> wrote:
Kent Watsen <kent+ietf@watsen.net<mailto:kent%2Bietf@watsen.net>> wrote:
>
>
> > On Oct 1, 2020, at 9:23 AM, Martin Björklund <mbj+ietf@4668.se<mailto:mbj%2Bietf@4668.se>> wrote:
> >
> > Kent Watsen <kent+ietf@watsen.net<mailto:kent%2Bietf@watsen.net>> wrote:
> >> Hi Qin,
> >>
> >>> Some opstate must be persisted, e.g., long-lived counters, logs, etc.,
> >>> but it’s a good point about other opstate not being persisted.
> >>> Perhaps “node-tags” can be used here, to differentiate which is
> >>> which…and servers can indicate if/how they support the ephemeral
> >>> opstate leafs in queries?
> >>>
> >>> [Qin]:That's a good case for node tag, in earlier discussion, we
> >>> discussed operation type, which distinguishs cumulative statistics
> >>> value from current value. The case discussed here is very close to
> >>> operation type proposal discussed earlier.
> >>
> >> Yes.  Thank you for pointing that out.  I meant to make the same
> >> observation before.  Indeed, such node-tags could have dual-purpose:
> >> to guide a streaming-strategy and a querying-strategy for certain
> >> nodes.
> >>
> >>
> >>> Note sure how others feel about “direction: (c), but my primary
> >>> use-case revolves around time-series data (e.g., logs), where the
> >>> interest is commonly on the most-recent entries, so
> >>> "reverse-->offset—>limit” works nicely.
> >>>
> >>> Perhaps an alternative would be to lift a concept from Python with
> >>> negative indexes so, for instance, offset=-N and limit=-N gives the
> >>> last N entries?
> >>> [Qin]: Yes, that's what I thought as well, with negative indexes, (b)
> >>> and (c) seems to me, can be combined.
> >>
> >> Can others comment on this?
> >
> > Isn't this just another syntax for the same function?
>
> No, it is not.

How so?  Isn't the idea that you can first ask for offset=-10, then
-20, etc, essentially walking the list backwards?  (I don't understand
what a negative limit means though).

Can somebody explain the use-case for iterating a list backwards?
[Qin]: See discussion with Robert
https://mailarchive.ietf.org/arch/msg/netconf/yRtWgDe7PUUznSvFTvhLYo7PpK4/
I feel if the client can remember the last element he request, he can use the last element as cursor anchor point
And iterate a list backwards by sending the second paginate request with negative offset.
The assumption for the server is the server should provide snapshot for all the paging data stored somewhere.
Relying on DB?
No customer has ever asked for this so I am wondering what we are all missing.
[Qin]: Yes, all our current customer ask for list forwards capability.

I guess it isn't clear that offsets do not work for lists where entries
are added and/or deleted over time,

[Qin]:how about lock configuration before sending paging request?
unless lots of state is kept by the server.
I guess if you are looking for the most expensive heavyweight solution possible
then this is a good start.



/martin

Andy



>
> K.
>
>
> >
> > /martin
> >
> >
> >> Presumably, we could eliminate “direction” (c) with this approach.
> >>
> >> Without “direction”, I think that UIs can still support the ability to
> >> do column-sorts, whereby the user clicks on a column’s header to
> >> toggle ascending vs. descending presentation, but they’ll have to do
> >> it client-side.
> >>
> >> That is, if wanting to see the 2nd page of results sorted by a column,
> >> something like:
> >>
> >>    sort(column-name) --> offset(-2*pagesize) --> limit(pagesize)
> >>
> >> Followed by the client then flipping the results to present the
> >> results in the user-selected order, right?
> >>
> >> That said, given that DB-backends that support sorts commonly also
> >> support direction, it's unclear what this buys us.
> >>
> >>
> >>>>> Sure, but I wonder if, e.g., a netmask filter, is supportable by
> >>>>> common DB-backends.  I’m hoping we have some DB-experts on the list!
> >>>>
> >>>> See above.  It can be quite efficient even if the backend doesn't
> >>>> support it.
> >>>
> >>> I don’t see that above, but I don’t doubt that it can be so, it’s just
> >>> a whole lot of implementation complexity.  It seems that we
> >>> should/must support servers doing it, we just need to find a way
> >>> (node-tags?) to enable them to express that ability.
> >>> [Qin]: My feeling is this efficiency more depends on the amount of
> >>> data we need to request. If amount of data we request is huge, maybe,
> >>> client-> server-> backend may be the better choice.
> >>
> >> Is it the amount of data requested or the number of entries in the
> >> list?  At least, in my worldview, clients are always requesting a
> >> “page” of data, so that part is rather consistently small.
> >>
> >> If the intention is to get a complete dump, then maybe the comment
> >> from yesterday applies, whereby streaming to an external repository
> >> that can be queried offline makes more sense?  - especially
> >> considering that the number of on-box logs is likely to be only the
> >> most recent (e.g., days), whereas the complete-dump type queries
> >> likely wish to extend well-past that.
> >>
> >> K.
> >>
> >>
>
_______________________________________________
netconf mailing list
netconf@ietf.org<mailto:netconf@ietf.org>
https://www.ietf.org/mailman/listinfo/netconf