Re: [Json] JSON Schema Language: extensibility and unspecified properties

Ulysse Carion <ulysse@segment.com> Mon, 12 August 2019 07:50 UTC

Return-Path: <ulysse@segment.com>
X-Original-To: json@ietfa.amsl.com
Delivered-To: json@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 77E58121375 for <json@ietfa.amsl.com>; Mon, 12 Aug 2019 00:50:55 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level:
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=segment.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eGjtFMDElovx for <json@ietfa.amsl.com>; Mon, 12 Aug 2019 00:49:51 -0700 (PDT)
Received: from mail-ot1-x341.google.com (mail-ot1-x341.google.com [IPv6:2607:f8b0:4864:20::341]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5CB8D120E3D for <json@ietf.org>; Sun, 11 Aug 2019 10:37:55 -0700 (PDT)
Received: by mail-ot1-x341.google.com with SMTP id g17so214898otl.2 for <json@ietf.org>; Sun, 11 Aug 2019 10:37:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=segment.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=94XiiS+Gky0C/HCh0d2QRgdnpDDzuNw9eptBChPDAh8=; b=el8jZ7ekMiL91lnHf+UcgdYOk9nNkU1vXv9nQjasPmc/Ai0HRQ1eLBs1v3ylLHWe75 OsXWWdSilP9Xcra4yE7Zxi+IYe1xvN4UAD/St56vEQn/QVvWr97ksKMFA3L30zqb7tTR xOmMIK0R4L/2Hb1GT23hFLxGGvrIKBYC5Gn/k=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=94XiiS+Gky0C/HCh0d2QRgdnpDDzuNw9eptBChPDAh8=; b=gRZMQuaJ13B1M6cB8BSYfo7u+pzDArIsqMHMC132B413d3tgqbv99Y2Ohd3ATal1Y9 H/pkJaj+W8l93KSwXtOuir3g2sGaOFER05dUfaFPsbMjrR5akvm+nDgtjuVLS3ioY7BD ArQ2f13nq/hbpJjkTR3a3QeelwoIOzGVf+lHiWOzgumoTcY92U9/oVSMEod0pYM/l8Sf +LzV0Y6ZUkd5uacnLV/dl9dowrVOBnqxBrgUEXA5r/KDNv0oYuok3WVPZsN1BIyA3nWI QN/2Ia6hyIFRZSd4G/EGP1E+xojclQ3CqEApV9GyaRUVFHQnpbcmi6Xn04G500+tMkMt 5hJg==
X-Gm-Message-State: APjAAAWH6KrO+kdE5IemIv/uQRkVPjRKk2f3b/u7G8EJmYy+3yiC2Mm5 MA5qEzk+NrhAQGvBWds6eTV9yL0+ydiwhWqgARBjerm4Jn4=
X-Google-Smtp-Source: APXvYqxLXcGMayAY2wNgWAUxUINvmVoWQ1Y6wlARYWJrJyOvKrK04jdRkfTnhyrdnFtBLyegBpkYwTO0NFH0lzwogHc=
X-Received: by 2002:a5e:d507:: with SMTP id e7mr18253052iom.284.1565545074631; Sun, 11 Aug 2019 10:37:54 -0700 (PDT)
MIME-Version: 1.0
References: <CAJK=1RhXp85cz-pOAQPw2JM=CYHgGSygj4Hw0spht56jbzQE2g@mail.gmail.com> <8d56f00d-14af-c387-ff02-e279cca6ddea@gmail.com>
In-Reply-To: <8d56f00d-14af-c387-ff02-e279cca6ddea@gmail.com>
From: Ulysse Carion <ulysse@segment.com>
Date: Sun, 11 Aug 2019 10:37:43 -0700
Message-ID: <CAJK=1RgxexzotnzfuaLqJ-r41=s2RzUEFJagVcYN82+i5s_MTQ@mail.gmail.com>
To: Anders Rundgren <anders.rundgren.net@gmail.com>
Cc: JSON WG <json@ietf.org>
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/json/PyBdfgy0nyhVEDhXyzRmYvEjpoQ>
Subject: Re: [Json] JSON Schema Language: extensibility and unspecified properties
X-BeenThere: json@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "JavaScript Object Notation \(JSON\) WG mailing list" <json.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/json>, <mailto:json-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/json/>
List-Post: <mailto:json@ietf.org>
List-Help: <mailto:json-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/json>, <mailto:json-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 12 Aug 2019 07:50:55 -0000

> How can you express this in a main schema using JCL?

I'm not entirely sure I follow your description. Could you illustrate
it with some example JSON values? I'm also a bit unclear on your
requirements for the "main schema", and how it fits with requirements
around "unspecified properties".

On Sat, Aug 10, 2019 at 12:00 PM Anders Rundgren
<anders.rundgren.net@gmail.com> wrote:
>
> If we stick to unspecified properties I'm currently working with a
> payment authorization scheme where some components will be defined
> by the payment networks.  The components start with a defined property
> holding a JSON Object but the contents of the object is not known by
> the "main" schema.
>
> In my programmatic solution the "main schema" ignores such components
> and only validates that they conform JSON.  The components are
> then dealt with by customer supplied decoders which indeed may use
> a schema.
>
> How can you express this in a main schema using JCL?
>
> Anders
>
>    paymentRequest = new PaymentRequest(rd.getObject(PAYMENT_REQUEST_JSON));
>    undecodedPaymentMethodSpecific = rd.getObject(BACKEND_METHOD_SPECIFIC_JSON);
>    rd.scanAway(BACKEND_METHOD_SPECIFIC_JSON);
>    referenceId = rd.getString(REFERENCE_ID_JSON);
>
>
> On 2019-08-10 03:43, Ulysse Carion wrote:
> > Hi folks,
> >
> > Thanks again for all the wonderful suggestions for JSON Schema
> > Language. I've just published a new draft incorporating suggestions
> > from the last update. You can read it here:
> >
> > https://tools.ietf.org/html/draft-ucarion-json-schema-language-02
> >
> > I would like advice on the two following changes I've made to the spec.
> >
> > First, I added explicit discussion of how JSL can be extended. The
> > language is at the end of section 2, but I'll reproduce it here:
> >
> >> This document does not describe any extension mechanisms for JSL schema
> >> validation. However, schemas (through the `non-keyword` CDDL rule in this
> >> section) are defined to allow members whose names are not equal to any of the
> >> specially-defined keywords (i.e. `definitions`, `elements`, etc.) described in
> >> this section. Call these members "non-keyword members".
> >>
> >> Users MAY add additional, non-keyword members to JSL schemas to convey
> >> information that is not pertinent to validation. For example, such non-keyword
> >> members could provide hints to code generators, or trigger some special behavior
> >> for a library that generates user interfaces from schemas.
> >>
> >> Users SHOULD NOT expect non-keyword members to be understood by other parties.
> >> As a result, if consistent validation with other parties is a requirement, users
> >> SHOULD NOT use non-keyword members to affect how schema validation, as described
> >> in {{semantics}}, works.
> >
> > Basically, my stance is: you can add custom stuff to schemas, but you
> > can't change validation. I expect that people will change validation
> > anyway, but they're not going to expect portability. I don't feel that
> > adding some sort of registry of keywords / forms is valuable yet.
> > Perhaps something like that can be in whatever replaces JSL?
> >
> > How do folks feel about this? Does anyone have suggestions on better
> > ways I can approach extensibility, or how I should phrase this
> > language?
> >
> > The other thing I want advice on is "strict" mode, called "strict
> > instance semantics" in the spec. Basically, strict instance semantics
> > about whether it's ok to send "extra"/"unknown"/"unspecified"
> > properties in an object.
> >
> > Right now, validation behavior is specified as a single, top-level
> > "strict": true/false (default is "true") property. You can't mix and
> > match strict and non-strict. But Carsten off-list noted that might be
> > a bit "blunt". Do others agree?
> >
> > The context for my decision is that two of major typed languages used
> > in industry, Java and Golang, don't typically support mixing and
> > matching. Go just as a blunt DisallowUnknownFields:
> >
> > https://golang.org/pkg/encoding/json/#Decoder.DisallowUnknownFields
> >
> > And Java's Jackson has a FAIL_ON_UNKNOWN_PROPERTIES:
> >
> > http://fasterxml.github.io/jackson-databind/javadoc/2.9/com/fasterxml/jackson/databind/DeserializationFeature.html#FAIL_ON_UNKNOWN_PROPERTIES
> >
> > Both are all or nothing.
> >
> > Also, there is the question of whether "strict" is a good name for
> > this property in schemas. Is there a better name for it? "strict" can
> > be understood in many ways. Open to ideas here as well.
> >
> > Other changes:
> >
> > - Removed int64/uint64 (and removed discussion related to I-JSON)
> > - Clarified that 1.0e1 is an integer
> >
> > Finally, I've moved JSL to a desired status of "Informative", and I'll
> > be going through the independent stream once the discussion here
> > appears to have settled.
> >
> > Thanks again for all the help,
> > Ulysse
> >
> > _______________________________________________
> > json mailing list
> > json@ietf.org
> > https://www.ietf.org/mailman/listinfo/json
> >
>