Re: [netconf] Current activities on RESTCONF/NETCONF to support paging

Douglas Hubler <douglas@hubler.us> Thu, 14 March 2019 20:08 UTC

Return-Path: <douglas@hubler.us>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4BE5E12785F for <netconf@ietfa.amsl.com>; Thu, 14 Mar 2019 13:08:56 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Level:
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=hubler-us.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eyqikA9bXkbT for <netconf@ietfa.amsl.com>; Thu, 14 Mar 2019 13:08:53 -0700 (PDT)
Received: from mail-it1-x12a.google.com (mail-it1-x12a.google.com [IPv6:2607:f8b0:4864:20::12a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A6A611277DE for <netconf@ietf.org>; Thu, 14 Mar 2019 13:08:53 -0700 (PDT)
Received: by mail-it1-x12a.google.com with SMTP id m137so6794856ita.0 for <netconf@ietf.org>; Thu, 14 Mar 2019 13:08:53 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hubler-us.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=E22qDmRjUthyZS70dBox4PEEQiL/TMZTqDWxoKKtiso=; b=w1eVT+pD0wTYOD7GSYetwcH43uZqqE3oaJR4U4WwuclhH9oDXHDRIF3JeTuYXOMUNx 7CA3mriFr1lL1am7OvFkzi02gjuwpFi415boyeQngQ68cAIxwiD0XZ92D+hU2H9cc6Ny hwa92vF5przQUl0bb3DA3kh4/Y7EaYeMscgwHcwQf2ugbW8cURKBpBBulCOOB6FEiAdV lHj6jxAujNqtAycAN3GIucUP5L5OpRQAQE6P82UhpeRG+CUYZ7QvbxiOYMTx0B9GX3wg dNAFSdTWhZbV78yP/JVhF3foQLjmLk+oj7aPhLXEbkWaarpTEiYxbjwlSnr+gZznYuV9 B5pg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=E22qDmRjUthyZS70dBox4PEEQiL/TMZTqDWxoKKtiso=; b=CBCbAhRvgaCxDbi8b3iSPz4cjfRc+0fZyPLlSn+80EQuxoYIpISn36CuIOu8B6GZMm dVzt21T0tn5hOL+HBgAuk0j3g/aADU4v0iaYgRgpyM27vMMZKvkWiaUBdJlIj81YNK2z jJrEpLA+XKsQjET1HmdcIVtwHxS7nrbcs6Q2ccMWwEGT81dFos5cFMFgL8esPGcRyB29 6bXEwZXEsHF7SvtWgVTaiPCdv+gA681mE7BspYESIdYWK79RSKpamhYBs8FaIDgtVFRA TNQqvVzpZgoJodU1InshTMxltcIhL2y9kOOirz/HnNbXJcE0AskUQTVm8n2XPTKiLP0Z /sQA==
X-Gm-Message-State: APjAAAVpzGgRN7MSfb6qdYnCT5Aut1Z/KqKHYqTBYj+tqiGlvH5scTpo u6086u81uR8y6ABKUTPTDYVEgMWsC6Q=
X-Google-Smtp-Source: APXvYqzb5nqX6cNBDfQxg86iQzIQPRjTtq667d+1R/DEj9nzETHt78m8QwWsTyFVytXTp1lIwfYSGA==
X-Received: by 2002:a24:ac58:: with SMTP id m24mr179707iti.129.1552594132799; Thu, 14 Mar 2019 13:08:52 -0700 (PDT)
Received: from mail-io1-f42.google.com (mail-io1-f42.google.com. [209.85.166.42]) by smtp.gmail.com with ESMTPSA id c25sm7129473ioa.75.2019.03.14.13.08.51 for <netconf@ietf.org> (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Mar 2019 13:08:52 -0700 (PDT)
Received: by mail-io1-f42.google.com with SMTP id x4so6293346ion.2 for <netconf@ietf.org>; Thu, 14 Mar 2019 13:08:51 -0700 (PDT)
X-Received: by 2002:a5e:8e45:: with SMTP id r5mr9636665ioo.221.1552594131694; Thu, 14 Mar 2019 13:08:51 -0700 (PDT)
MIME-Version: 1.0
References: <C88A38CA-44B5-42A0-9DFA-A67AE5456951@nokia.com>
In-Reply-To: <C88A38CA-44B5-42A0-9DFA-A67AE5456951@nokia.com>
From: Douglas Hubler <douglas@hubler.us>
Date: Thu, 14 Mar 2019 16:08:40 -0400
X-Gmail-Original-Message-ID: <CALAkb6cMg8nKKW9yCw0SYVE=zO1Az_02fCi7JbUW9enO75xiQg@mail.gmail.com>
Message-ID: <CALAkb6cMg8nKKW9yCw0SYVE=zO1Az_02fCi7JbUW9enO75xiQg@mail.gmail.com>
To: "Wisotzky, Sven (Nokia - DE/Stuttgart)" <sven.wisotzky@nokia.com>
Cc: "netconf@ietf.org" <netconf@ietf.org>
Content-Type: multipart/alternative; boundary="000000000000b33a610584137d21"
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/e1wxuTbPeP5uwDI6RGRrEoeHOVQ>
Subject: Re: [netconf] Current activities on RESTCONF/NETCONF to support paging
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Mar 2019 20:08:57 -0000

On Thu, Mar 14, 2019 at 5:24 AM Wisotzky, Sven (Nokia - DE/Stuttgart) <
sven.wisotzky@nokia.com> wrote:

> There was some discussion in earlier days (back in 2012/2014) to support
> paging for extra-long lists. The main use-case is clearly to improve
> support for interactive user-frontends, as loading entire lists with
> potentially more than 100.000’s objects easily can become a nightmare:
> loading time, memory consumption, crashes, …
>

speaking from experience, RESTCONF is very usable straight from interactive
user-frontends and this is one of the key things missing.  I did add a
custom parameter to FreeCONF's RESTCONF implementation to support this.
Maybe I took this took far, but I decided that because data is in a
hierarchy you might have multiple lists you need to page thru so I came up
this this parameter

   range=b/c!N-M

returns rows N thru M inclusive in list under path b/c.  And that you can
specify multiple ranges for multiple paths.  Front ends usually have no
problem remembering what number they are on to track N and M.

As brought-up, for lists that are highly dynamic, this strategy is bit
naive and maybe etags can help the clients that need to know when list has
changed and they might need to start over.  For my backend, I could start
iteration at row N without have to manage a cursor.


I did not found any active DRAFT or RFC for this. That said, we can see
> various vendors implementing proprietary solutions for this in their
> RESTCONF stacks.
>
> Would really like to see an IETF defined approach for this, to avoid
> multi-vendor clients need to deal various vendors implementations.
>

+1