Re: [core] Separate formats for fetch and patch (was: WG Last Call of draft-ietf-core-senml-etch-03.txt)

[Christian M. Amsüss <[e-mail]>]

Hello CoRE,

on the interim of two weeks ago we had discussed writing something about
media types usable with FETCH and PATCH methods.

I've assembled a few lines -- I'm not sure whether they're good for
anything but a starting point for further discussion.

Unlike in the interim, I'm not convinced any more that reusing the
target media type for the fetch or patch content format has merit; it
may only make sense for very few media types where the fetch/patch
semantics are immediately obvious.

Best regards
Christian


On media types for FETCH and (i)PATCH
=====================================

Media types for the request payloads of FETCH, PATCH and iPATCH
operations are often not as straight-forward as the PUT request and GET
response media types. Several patterns have been deployed or considered:

Dedicated query/patch languages
-------------------------------

Request payloads use a media type that bears little to no resemblance to
the media types obtained by GET or as FETCH results, but use a dedicated
query language expressed in its own media type.

Examples:

* [SPARQL-QUERY], [SPARQL-UPDATE]: A query language in the style of SQL
  for RDF databases. The same grammar (with different entry points) is
  used for queries and updates, and distinct media types are described
  for the two applications. 

* XPath patches ([RFC5261]): An XML format that lists individual
  additions, replacements and removals to be enacted on a target
  document. [RFC7351] serves up an actual media type for that operation.

The examples have in common that while they use different languages,
they can contain data that will end up in the same format as the target
resource, but embedded in container language. (Xpath patches contain the
new values as part of their XML graph, SPARQL allows to have startements
that look just like Turtle).

Note that those examples, in practice, do not use the FETCH but the
POST or GET method (the latter with query strings), due to the absence
of a FETCH in HTTP; even for PATCH cases a POST is used as [SparqlPatch]
was not followed up on.

[SPARQL-QUERY]: https://www.w3.org/TR/sparql11-query/#mediaType
[SPARQL-UPDATE]: https://www.w3.org/TR/sparql11-update/#mediaType
[SparqlPatch]: https://www.w3.org/2001/sw/wiki/SparqlPatch
[RFC5261]: https://tools.ietf.org/html/rfc5261
[RFC7351]: https://tools.ietf.org/html/rfc7351

Query/patch languages that are almost valid states on their own
---------------------------------------------------------------

The request payload is syntactically very similar to the typically
obtained responses, but uses a different media type:

* SenML fetch/patch [etch]: The SenML format does not allow for values
  to be missing or null, but no value can be given in a query or a patch
  used to remove a record. A superset of the original language was
  expressed that removes the limitations, resulting in a format usable
  for fill-in-the-gap queries and patch-by-overwriting

* JSON [merge-patch] documents are used to patch arbitrary JSON by
  presenting the subset of the target document that needs change with
  the new values filled in, using the value null as a sentinel for "that
  part should not be there any more at all".

[etch]: https://tools.ietf.org/html/draft-ietf-core-senml-etch-04
[merge-patch]: https://tools.ietf.org/html/rfc7386

Fill-in-the-gap / patch-by-overwriting
--------------------------------------

The request payload has the same format as queries, but contains gaps
(in the case of FETCH) which it implicitly asks the server to fill, or
different values in some places.

* SenML fetch/patch was originally planned to work this way.

* [CoRAL] could be used in this fashion by asking the server to fill any
  blank nodes in the request tree with actual URIs if it has them and
  returning the subset of the tree it could show to be true (for
  fetching) and to add statements (but not easily to remove them) by
  forming a union graph.

A limitation of this approach is that it allows for only (exactly) one
mechanism to be applied per content type. That may be suitable if there
is only one way to "apply" or "fill in" a representation; any changes to
that mechanism will then require a new media type, even if the
descriptive format did not change.

[CoRAL]: https://tools.ietf.org/html/draft-hartke-t2trg-coral-09

General issues
--------------

An issue to be concerned about, whenever patching happens on generic
target media types, is the selection of the representation to be patched
(this does not apply to FETCH because there is an Accept involved
there): An XPath patch, for example, could be tried both against an
application/senml+xml representation and an vnd.example/vnd.sensor that
can conceivably exist of the same resource, leaving the server unable to
decide where to apply it.

Specifying in the content type of the patch against which media type it
is valid (might look like
application/merge-patch+json;target=application/link-format+json, but
that's currently not specified) may be a way around this, but that
brings with it multiplicative use of the media type code points we have
in CoRE.