Re: [core] Short review of SenML fetch/patch

Ari Keränen <ari.keranen@ericsson.com> Sun, 04 November 2018 19:52 UTC

Return-Path: <ari.keranen@ericsson.com>
X-Original-To: core@ietfa.amsl.com
Delivered-To: core@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D266D126CB6 for <core@ietfa.amsl.com>; Sun, 4 Nov 2018 11:52:28 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.77
X-Spam-Level:
X-Spam-Status: No, score=-4.77 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.47, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com header.b=Xsl0xeMX; dkim=pass (1024-bit key) header.d=ericsson.com header.b=GnK8Ijus
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2neq05BRhzTR for <core@ietfa.amsl.com>; Sun, 4 Nov 2018 11:52:26 -0800 (PST)
Received: from sesbmg22.ericsson.net (sesbmg22.ericsson.net [193.180.251.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EFA69123FFD for <core@ietf.org>; Sun, 4 Nov 2018 11:52:25 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; d=ericsson.com; s=mailgw201801; c=relaxed/simple; q=dns/txt; i=@ericsson.com; t=1541361144; x=1543953144; h=From:Sender:Reply-To:Subject:Date:Message-ID:To:Cc:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=nRMdPAttX5gxwHOA7CcRUztSYI76TzMIGYtoNDsQbOU=; b=Xsl0xeMXefDRuvpYM+sRq+UWqsygsLDA7+OwuJg/sCiprXIwjIzdhkKCbxLcxJ3w pYvzU6ldiUYuBvHp6q57c+n8FeU/jAy1s7rB3OmhVotPJJSB8z/YCXa6NZlxoQNk BKxHUAgSOpWjmRWYxm3fSs/LbnKZVu4ZbuSP63O1u/Y=;
X-AuditID: c1b4fb30-1ebff70000007d19-cd-5bdf4df78cff
Received: from ESESBMB502.ericsson.se (Unknown_Domain [153.88.183.115]) by sesbmg22.ericsson.net (Symantec Mail Security) with SMTP id 0E.33.32025.7FD4FDB5; Sun, 4 Nov 2018 20:52:24 +0100 (CET)
Received: from ESESBMR505.ericsson.se (153.88.183.201) by ESESBMB502.ericsson.se (153.88.183.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1466.3; Sun, 4 Nov 2018 20:52:23 +0100
Received: from ESESBMB505.ericsson.se (153.88.183.172) by ESESBMR505.ericsson.se (153.88.183.201) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1466.3; Sun, 4 Nov 2018 20:52:23 +0100
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (153.88.183.157) by ESESBMB505.ericsson.se (153.88.183.172) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1466.3 via Frontend Transport; Sun, 4 Nov 2018 20:52:23 +0100
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K4/lPIlnfP5X8ZKZps/04KoN/9Xi/bQShLNmLP8lOII=; b=GnK8IjustQcRrWv+Va1NN5eCFgHzXU3Oke7Nu6lDjXtivLNyi6lOglIGAv6a53j0DuSvTZqVGmR/FYX+XG2KqVWEMeJWuv3F7OoxM2DJ58bMddXg43SceTivFcDHOM2Z23DODWYSHngdNdcYtFYW03ka9YhLIB+Da/CpLMe0W5s=
Received: from HE1PR07MB4236.eurprd07.prod.outlook.com (20.176.166.145) by HE1PR07MB3356.eurprd07.prod.outlook.com (10.170.247.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1294.10; Sun, 4 Nov 2018 19:52:22 +0000
Received: from HE1PR07MB4236.eurprd07.prod.outlook.com ([fe80::b074:afe2:469c:dd95]) by HE1PR07MB4236.eurprd07.prod.outlook.com ([fe80::b074:afe2:469c:dd95%3]) with mapi id 15.20.1294.032; Sun, 4 Nov 2018 19:52:22 +0000
From: =?iso-8859-1?Q?Ari_Ker=E4nen?= <ari.keranen@ericsson.com>
To: =?iso-8859-1?Q?Christian_Ams=FCss?= <christian@amsuess.com>, core <core@ietf.org>
Thread-Topic: Short review of SenML fetch/patch
Thread-Index: AQHUVkB5+PFf3+Mfc0qMr/ag7hWmPw==
Date: Sun, 4 Nov 2018 19:52:22 +0000
Message-ID: <HE1PR07MB42369A7652E57724E095302185C90@HE1PR07MB4236.eurprd07.prod.outlook.com>
References: <20180927085947.GA5321@hephaistos.amsuess.com> <8F814ABD-10B3-4279-945B-D1B33E0BD1A8@ericsson.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is ) smtp.mailfrom=ari.keranen@ericsson.com;
x-originating-ip: [89.166.49.243]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; HE1PR07MB3356; 6:HUdkwyQXAlYqe7B0AAPthCboGCnfV8s/CSSD+upgfTIUitDII9m5Owh8ncXGhfDZzyszMsOFmsq2/ntQBaT3ZtdGWbgM6FYq9dHzDT2L+ZEHGnwgPGewmCy/l+anDJhgz2rnp+Qy0AHt/2CEL+dA9/QNK0Zrl5iT2AOxEioPjIqPVVzkk2ZK6IQBX+LiPtk/brp20fTDKT2ht4SR3opKklh9JkzqyCCEFmks0+Kwt0eiOpU8LPd6Tc5a3ZTvKFTrloSbOVUwUXgBAW0XuK22sTw4Nb2u0p96ZvZZ1ocTzPUTaKnxECMa3gylYuXFpxo8GCRyqmoDQ/OtYjD7jPAn0UL4k+y+bCiSilnxK/PT+fCF1TmTRBVOEYalt0WUExaf+N6qL+nVx1QTGqs+NPSGcUPSBcX7unisuU7FOp1THcnvdmWke35ucEKzvxE0dyn2XZEt/JaIpQP9zpVIWBaCpA==; 5:t8Ys0QotdGQ+tIR6Nkd+DIBYGMIw7DCs4K60g/sNjrXrwUECpi3lcCO+/HMlFrTQwDr8169imS0UREgTV+cWNDjVoDHCZUSzijPbJkFcXGDQFI1x+NWOSDGxoIBl87aVj5Sj2EXGN+EajDA5bLmsArdgV4u25L8b/+KM3XTs5jw=; 7:0NUo3HXAxtYs8jEwN16iqthGEfPj/lMcMMNQHAekoHpSBOiEESJ4vjHWyK8sj036xPsodlWX+TzTTpLQD58p8ImnMZczk8M95oHZMDD8vR8iIORH0OgyJ8TRj30351VizgSBuQ54TPxRcI82ryNo1A==
x-ms-exchange-antispam-srfa-diagnostics: SOS;
x-ms-office365-filtering-correlation-id: 11b53a08-297d-4ae1-8680-08d6428f0934
x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(2017052603328)(7153060)(7193020); SRVR:HE1PR07MB3356;
x-ms-traffictypediagnostic: HE1PR07MB3356:
x-microsoft-antispam-prvs: <HE1PR07MB335633A1BD3E753E759BB42285C90@HE1PR07MB3356.eurprd07.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(278428928389397)(192374486261705)(17755550239193);
x-ms-exchange-senderadcheck: 1
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3231382)(944501410)(52105095)(93006095)(93001095)(10201501046)(3002001)(148016)(149066)(150057)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123562045)(20161123564045)(20161123560045)(201708071742011)(7699051)(76991095); SRVR:HE1PR07MB3356; BCL:0; PCL:0; RULEID:; SRVR:HE1PR07MB3356;
x-forefront-prvs: 084674B2CF
x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39860400002)(396003)(136003)(366004)(376002)(346002)(199004)(43544003)(189003)(74316002)(256004)(71190400001)(305945005)(71200400001)(5660300001)(26005)(66066001)(14444005)(229853002)(345774005)(97736004)(81156014)(6246003)(81166006)(8676002)(105586002)(8936002)(25786009)(106356001)(86362001)(6436002)(6506007)(53936002)(76176011)(2906002)(316002)(68736007)(478600001)(7696005)(55016002)(2900100001)(99286004)(486006)(7736002)(446003)(476003)(66574009)(186003)(102836004)(6116002)(3846002)(33656002)(6306002)(9686003)(110136005)(14454004)(966005); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR07MB3356; H:HE1PR07MB4236.eurprd07.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1;
received-spf: None (protection.outlook.com: ericsson.com does not designate permitted sender hosts)
x-microsoft-antispam-message-info: LnG3bD8TZ77DBefEcQ0Wl+vJWoUEMHuIxtj/rXGnLdzcHJQPkYL99DHTeCD7bFRQYx/k+2uUp7ayVMFcw4CA00oZ1AqKTTaqsf5R2+0myKat+w1owlGQIarlpo8vGnTt2zGMyFm30JVUtYu9ToPTCX1WM8tboFZhC6mhpRI+Lnpsc7qBgWJ+zIhkF3uMJ7CQmdmz/SaxohSudj7zlFGLD7I1KuqV8FvrcPyu10SamlxxoOrxHLKf5Hkl9kM/mPqi53SOXJC0tcQ6LEJZZDsc6K54V5+ojSmpwJ4DkXfsgdf97gHSeKKV/RtkiXpMhTYiy5+ZhmD+iaoySbXB8hd3ScLOgMGluPNrzlPDfjCxVQI=
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 11b53a08-297d-4ae1-8680-08d6428f0934
X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Nov 2018 19:52:22.0536 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR07MB3356
X-OriginatorOrg: ericsson.com
X-Brightmail-Tracker: H4sIAAAAAAAAA02Sa0hTYRjHec9lHoej4/LyoHRxCYmXuaTBIMmuaIhR2EVU0JUH71POMVH7 soqMJtryEirIrDa3xLB05T6EuCWoAxNHjRTBhpI3FBUyW6RtOwv69n9////zvM/z8lK4+AEZ QRWrqhhWpSyTCIRER9YQl7CbMZ8j205VDFuaCMXwej9+BkvTWN1kml7/C7uCZQuTC5iy4mqG TTydLyzq+zwtqJyR1zyz7GBqNBCnQYEU0CfhXfPjAA0SUmJ6FMFS8wbyGmL6B4JH7Xm84dE9 /Q6MN15i8MoY4zUIWovD9icj4lMtGJht7zH+4EKgW3P6SgR0CrT0bXlSFBVC34QnhjAvPkhL wd3xh+RxIjgnMr04xIN3J5dxryboaFi/v+zrIqJzwazt9A9RCQbHiMCrER0GP+19Po7T4TC7 qMP41WjQf5jCeR0KKwt7JJ/PgVX7RADPo2Byw+XPHwKHrsG3C9BOAbhNzYg3EmCzrc3fKAPe Dg0SfGgMgXtjh+SNWFgZ7PIXlEJrl47gdS7MGxb9NxyG3kaXn4/h4GxM0CJZ53+D81oKX9ta BbyOg57na3in7wGCYaJjkehGRC8K5RjuVnlhUpKUYYtvc1yFSqpiqgaQ52tYzb9lFrSydNaG aApJgkRXT83niEllNVdbbkNA4ZIQ0ccLHiQqUNbWMWxFHnunjOFsKJIiJOEixeXBbDFdqKxi ShmmkmH/uRgVGKFGhlR7+viIlNkfJbeS444ELWSVRKXJSc2llO6YSKMjfkr9XdWUybD1Dees X/LtsBesUArPv1DHD02a5pKPjt9zWb+V9LA1N7SCA5LoJPnsHLEercice31x6ql5uuFht2LG FFl37PjItqlcf23/ut7YLt+sv/vGkrKavizTNkoIrkh5IhZnOeVfBZ4MvxYDAAA=
Archived-At: <https://mailarchive.ietf.org/arch/msg/core/kWS4WjlgwLSIkDuKTG7pSz2doGE>
Subject: Re: [core] Short review of SenML fetch/patch
X-BeenThere: core@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Constrained RESTful Environments \(CoRE\) Working Group list" <core.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/core>, <mailto:core-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/core/>
List-Post: <mailto:core@ietf.org>
List-Help: <mailto:core-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/core>, <mailto:core-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 04 Nov 2018 19:52:29 -0000

Hi Christian et al,

Thank you once more for the thorough review Christian!

After discussing the media type issue with various CoRE and OMA DM 
folks, using new media types for the SenML FETCH/PATCH seems to be a 
good approach. Hence I'm proposing we'll go that way in the next version 
of the draft (see -00 draft [1] how that could look like in practice). 
This should address most of your remaining comments below.

We still need to decide on how to indicate delete operation though. "v: 
null" and "vdel: true" seem to be our options. Some pros and cons are 
discussed in the original e-mail below. Input on this is very welcome.


Cheers,
Ari

[1] https://tools.ietf.org/html/draft-keranen-core-senml-fetch-00#section-5

On 28/09/2018 1.07, Ari Keränen wrote:
> Thank you for the review Christian!
> 
> See inline.
> 
>> On 27 Sep 2018, at 11.59, Christian Amsüss <christian@amsuess.com> wrote:
>>
>> Hello authors,
>>
>> as requested during yesterday's interim meeting, I've had a look at
>> senml-fetch again:
>>
>> Fetch
>> =====
>>
>> Fetching by time
>> ----------------
>>
>> Being able to pick aparticular timestamp is an oddly specific
>> application; my impression of general SenML data sets is that it will be
>> hard to predict which point in time should have a datum if that record
>> is not already known. Unless there are good (and argued for) use cases,
>> I suggest that the topic of fetching time series points be explored
>> comprehensively in a separate document, possibly hooking into
>> draft-bormann-t2trg-stp.
> 
> The time-stamp use case I had in mind was where the record is already known, so it's mainly useful for PATCHing, but didn't see issue having it for FETCH too (say, you know time out of band). Otherwise we have no mechanism to tell apart two records from the same source.
> 
> 
>> Using SenML to fetch
>> --------------------
>>
>> SenML requires to have a value (or sum) on every record, the payload of
>> the fetch request therefore can't use any of the defined SenML mime
>> types but would need to specify their own. I'm unsure on whether adding
>> a field like "fetch_":1 to the first record (a mandatory-to-decode
>> custom field) would be sufficient to work around that constraint.
>>
>> Otherwise, it might be an option to update RFC8428 to say that some of
>> those limitations (also the later "v":null) may be overridden in
>> applications like this where it is made sure (eg. by never using those
>> freedoms in Content or PUT payloads) that no generic SenML processor
>> ever gets its hands on the document.
> 
> Good point. Maybe updating 8428 with the exception that restrictions on value don't apply for FETCH/PATCH is appropriate.
> 
>> The payload sent in as a FETCH payload is not actually sensor data, it's
>> a list of names (or, provided they're viewed as URIs, which I recommend)
>> links. We already have a way to transmit a list of links in CoRE
>> (actually 3 with links-json). I don't object to introducing another
>> (given the above, might be called application/links+senml+{cbor,json}?)
>> especially to reduce parsing complexity, but do suggest that FETCHing
>> SenML data be done with a list of identifiers no matter how they are
>> transported, and the recommended way can well be a SenML-names-only
>> payload (but would not preclude others).
> 
> The payload is a list of names without URI semantics (e.g., may not have scheme at all, have different concatenation semantics, etc.) and timestamps, so I think it's quite different from the other link formats.
> 
>> Patch
>> =====
>>
>> "value field with value null": There has been some backpressure against
>> having variable-type fields in SenML back when its JSON/CBOR structure
>> was last changed; having a "v":null in there could be problematic (esp.
>> as its JSON type is fixed to Number in Table 2 of RFC8428). I suggest
>> having a explicit "delete value" field, eg. "vdel":true.
> 
> The "v":null approach made this nice and clean, but you do have a valid point to consider here. Will have a closer look at this.
> 
>> How would base values work with patch? I figure that it's always the
>> resolved record that matter, but that should be explicit. In connection
>> with a "vdel" (or "v":null if it turns out it's OK for it to stay like
>> that), it might make sense to allow that too (where a "v":n would
>> override "vdel" or "v":null from an offset of zero) to allow for
>> efficient batch removal of values.
> 
> Yes, resolved records are that matter (see end of section 3.1). I'm not quite sure what you suggest allowing though?
> 
>> Security considerations
>> =======================
>>
>> "potentially allow retrieving or changing many resources at once" makes
>> it sound like FETCH and PATCH could reach out to different resources.
>> They don't (conceptually), they only manipulate what's inside the target
>> resource (which typically is an aggregation of other resources, as in a
>> core-interfaces batch). Suggested replacement:
>>
>>    In FETCH and iPATCH requests, the client can pass arbitrary names to
>>    the target resource for manipulation. The resource implementer must
>>    take care to only allow access to names that are actually part of (or
>>    accessible through) the target resource.
>>
>>    If the client is not allowed to do a GET or PUT on the full target
>>    resource (and thus all the names accessible through it), access
>>    control rules must be evaluated for each record in the pack.
>>
>> (It's fine if someone goes ahead and defines a /senml resource on their
>> device that is a gate to "everything and anything that can be addressed
>> in SenML", but that shouldn't be the default assumption).
> 
> Good point. The replacement looks good to me.
> 
> 
> Thanks,
> Ari
> 
>> Best regards
>> Christian
>>
>> -- 
>> You don't become great by trying to be great. You become great by
>> wanting to do something, and then doing it so hard that you become great
>> in the process.
>>   -- Marie Curie (as quoted by Randall Munroe)
> 
>