Re: [netconf] restconf collections

Hongwei Li <flycoolman@gmail.com> Fri, 02 October 2020 13:31 UTC

Return-Path: <flycoolman@gmail.com>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C02B53A1623 for <netconf@ietfa.amsl.com>; Fri, 2 Oct 2020 06:31:39 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Level:
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vc79eMK_Le4A for <netconf@ietfa.amsl.com>; Fri, 2 Oct 2020 06:31:37 -0700 (PDT)
Received: from mail-lj1-x22f.google.com (mail-lj1-x22f.google.com [IPv6:2a00:1450:4864:20::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 952303A1622 for <netconf@ietf.org>; Fri, 2 Oct 2020 06:31:36 -0700 (PDT)
Received: by mail-lj1-x22f.google.com with SMTP id w3so1196472ljo.5 for <netconf@ietf.org>; Fri, 02 Oct 2020 06:31:36 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Bw7P7ua/zfcwZzuvOY5nH/Y38mZyODr7Y8RVRXqNu1A=; b=JA9FCKIplswjh2meTMRHpQRdZcZHdMcMkZZBJyTJoi3W32RRe0DZajITvsxDPFOVkc RdH4jQL//KrAkdfnsX+QbdLfsABNsGJ+JeLs/SxOamcfHjzZf+hRloGo6ZiyGxlBPImT VwPaPWMK0hBBjkuD358Mo/jRkScyS4VMaVG9O3CQ6ft66qXFHwMl/l+Wdgrnf+t9doZY x8Hvb9iMUknJ4KFq914rAl9jf/IWYq8ASnXTwyHwURy0bijl+mIMiYzthmeK+VgD3x4e 5a+wZ2KVfsceAlOIyI2W0CmVrB5QWUZ/K9t8hmzgLuz9OqVQg6G6U917sGxJi3lnSLyC 6POA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Bw7P7ua/zfcwZzuvOY5nH/Y38mZyODr7Y8RVRXqNu1A=; b=UXspFQHPSv+9NZSrSkefkHw6l6C9PXrsriJeSdElvzc1hZyaQpDS6wCJJCc0CvI0yH a4B7cvpvk+dclLviK8rTDAPhdox9Bl3kEUkfx7zrLM/3Us8Pdd/5zyhWA5XLQXRo8izn w8BAnNCCWggBl1F3YjUZWbCAT7YdwYs4MXxTFXjLurbstkbKqrxlGHHE+Eveuzs4Vcdo xDRiG5HRgai+i2SDYwTVZg5Gw7+qni5ot4n0G5BxlpSN63R9Akcztq+2DvlhvKWFkGQI f86LXPIYnxXZy6gZZ9KmpZi9rAo0KELmpOVp/o1Ekh7SvbBP5a7yTL7f5qWuloOGsuPv WmBg==
X-Gm-Message-State: AOAM533HQ1/rUALXkbRtckLiSPoxlfXyqIhU0y+zz37N9xDT0xP/pnch ktPXM97I/A7/74BvbW50W+j8i1FiYAVKM+vpRA0=
X-Google-Smtp-Source: ABdhPJyKo+jV620aG9kKUZYOR43SLfGSTBEs4nQKVmn5ccBgqj6fuIrrPu1LzLrVGRyh5/vagGuaUI+YIP9lq4BpniY=
X-Received: by 2002:a2e:808f:: with SMTP id i15mr675910ljg.51.1601645494628; Fri, 02 Oct 2020 06:31:34 -0700 (PDT)
MIME-Version: 1.0
References: <01000174e4610ae6-d1488376-ae7e-4d95-ba7e-005fc44b9592-000000@email.amazonses.com> <20201001.165548.1275939966226069939.id@4668.se> <01000174e5ba8cc5-39edb446-48d9-4896-a6b5-dcf80f962762-000000@email.amazonses.com> <20201002.082331.2242297379846191383.id@4668.se> <01000174e96cd06f-4ecb06ac-c7ac-4233-bd5d-97d3dbc74830-000000@email.amazonses.com>
In-Reply-To: <01000174e96cd06f-4ecb06ac-c7ac-4233-bd5d-97d3dbc74830-000000@email.amazonses.com>
From: Hongwei Li <flycoolman@gmail.com>
Date: Fri, 2 Oct 2020 09:31:22 -0400
Message-ID: <CAL73O_x_mPwB6S-=UdASFK_H9f7CkeYU270ZZHX5xf9eLM=iAQ@mail.gmail.com>
To: Kent Watsen <kent+ietf@watsen.net>
Cc: =?UTF-8?Q?Martin_Bj=C3=B6rklund?= <mbj+ietf@4668.se>, "netconf@ietf.org" <netconf@ietf.org>
Content-Type: multipart/alternative; boundary="000000000000c375fb05b0b02646"
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/nvzdjad9dOHYc0CfTkNHh6BlGrg>
Subject: Re: [netconf] restconf collections
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 02 Oct 2020 13:31:40 -0000

Hi Kent,


On Fri, Oct 2, 2020 at 9:07 AM Kent Watsen <kent+ietf@watsen.net> wrote:

>
> 3) Here we’re talking about querying a potentially large repository of
> logs that are at rest, and full table scans should avoided.  I suggest
> that we’re dealing with something new here.
>
>
> Perhaps we don't understand each other.  Here's an pseudo-code example
> of what I envision:
>
>  GET /interfaces/interface
>  filter  'oper-status = "up" and statistics/in-errors > 0'
>
> This is XPath 1.0, applied to one instance at the time.
>
>
> Of course I understand Xpath filters, other relevant examples here:
>
> - https://tools.ietf.org/html/rfc5277#section-5.2
> - https://tools.ietf.org/html/rfc8040#appendix-B.3.6
>
> The point was/is that supporting full-Xpath for stream-filters is easy, or
> rather doesn’t entail special consideration, as already each event is in
> memory for an Xpath analysis.  Thus, pointing to existing usage as evidence
> that unconstrained Xpath can be used everywhere, including for potentially
> very-large CF-lists, is not a given, as it would be performance-limiting to
> bring every list-entry into application-level memory in order to use an
> Xpath library.  In this case, it is very much desired to push all the
> querying logic to the DB-backend.
>
>
> But when it comes to CF data, doing a full table scan is impractical
> and index-backed queries can make the difference between the client
> waiting seconds vs minutes.  This is why I was initially trying to
> focus on just solving the CF problem, where the pain is acute.
>
>
> Well, it is less likely that an implementation uses a feature-rich (in
> terms of query language) DB to store all its state, than its config.
> So it would probably be easier to implement this using "an index" for
> config than for config false...
>
>
> Really?  ...my experience is seems opposite.
>
> A number of server implementations I’m aware of persist the entire
> configuration as a flat-file (e.g., pretty-print XML or JSON).  They bring
> it into memory and act on it using a feature-rich library (e.g.,
> Yangson).  In my view, this approach hits the low-bar in terms of what
> constitutes a “database”.
>
> On the flip side, handling large CF-lists (e.g., logs) is where servers
> pretty much have to use a real database.  An RDB (relational DB) is common,
> but a TSDB (time-series DB)
> <https://en.wikipedia.org/wiki/Time_series_database> is better suited.
> [HL] A user case for time series data (e.g., logs), customers want to get
> the data in between Time A and Time B. Do we use timestamp filter here?
>


> Perhaps we’re saying the same thing (yes, config APIs tends to have better
> query expressions)…it might just be your last line regarding “index” that’s
> throwing me (as clearly the DBs storing large CF-lists are having indexes).
>
>
> That said, I propose we do “both”.  <snip/>
>
>
> I agree.
>
>
> Okay, we’ll shoot for a two-pronged approach.
>
>
>
> Separately, for the RC-list draft only, the plan is to declare “list” and
> “leaf-list” nodes (not their entries) as data resource targets.  We only
> need to do this for GET, but I assume that it's better to cover all the
> HTTP operations - thoughts?
> [HL] Data resources should not be bound to some specific operation.
> Correct me, if my understanding is wrong.
>
> K.
> _______________________________________________
> netconf mailing list
> netconf@ietf.org
> https://www.ietf.org/mailman/listinfo/netconf
>