Re: [Json] JSON merge alternatives

"Manger, James" <> Thu, 20 March 2014 00:35 UTC

Return-Path: <>
Received: from localhost ( []) by (Postfix) with ESMTP id 25F0F1A0831 for <>; Wed, 19 Mar 2014 17:35:31 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -0.202
X-Spam-Status: No, score=-0.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_AU=0.377, HOST_EQ_AU=0.327, RCVD_IN_DNSWL_NONE=-0.0001, RELAY_IS_203=0.994] autolearn=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id NSKe_Pz6zGZO for <>; Wed, 19 Mar 2014 17:35:28 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id C07281A0833 for <>; Wed, 19 Mar 2014 17:35:27 -0700 (PDT)
X-IronPort-AV: E=Sophos;i="4.97,690,1389704400"; d="scan'208";a="202824955"
Received: from unknown (HELO ([]) by with ESMTP; 20 Mar 2014 11:35:06 +1100
X-IronPort-AV: E=McAfee;i="5400,1158,7382"; a="261937622"
Received: from ([]) by with ESMTP; 20 Mar 2014 11:35:06 +1100
Received: from ([]) by ([]) with mapi; Thu, 20 Mar 2014 11:35:06 +1100
From: "Manger, James" <>
To: Nico Williams <>, "" <>
Date: Thu, 20 Mar 2014 11:35:05 +1100
Thread-Topic: [Json] JSON merge alternatives
Thread-Index: Ac9DzWMzeAxeC3EtScm9zkauyIhg8QAAHcZw
Message-ID: <>
References: <20140319234549.GA3471@localhost>
In-Reply-To: <20140319234549.GA3471@localhost>
Accept-Language: en-US, en-AU
Content-Language: en-US
acceptlanguage: en-US, en-AU
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
Subject: Re: [Json] JSON merge alternatives
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "JavaScript Object Notation \(JSON\) WG mailing list" <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 20 Mar 2014 00:35:31 -0000

RFC 6902 "JavaScript Object Notation (JSON) Patch" looks quite close to proposal 2.
Media type: application/json-patch+json
Operations: add, remove, replace, move, copy, test

draft-ietf-appsawg-json-merge-patch "JSON Merge Patch" is a simpler subset of proposal 1. It is designed to work well with JSON objects, and objects nested in objects etc. Its main limitation (compared to proposal 1) is that it doesn't support insert/replace for individual array elements.

Adding a "merge" operation to RFC 6902 would be another way to get the best of both these approaches.

James Manger

-----Original Message-----
From: json [] On Behalf Of Nico Williams
Sent: Thursday, 20 March 2014 10:46 AM
Subject: [Json] JSON merge alternatives

First, IMO the WG should in fact adopt a WG item (or more) for MIME
types for use with PATCH to patch JSON texts (or entities with JSON
representations).  This is commonly known as "JSON merge", IIUC.

I work with one Rails application that has an ad-hoc JSON merge like
schema for representing updates (PUTs) to resources -- this schema is
very specific to the application in question, therefore not reusable.

I can imagine a Ruby gem that handles a JSON merge PATCH in a completely
general and reusable way.  And not just Ruby, of course.

Second, I believe the JSON merge proposals I've seen to date are no

The goal of any Internet JSON merge schema should be to:

   Concisely and generally express "edits" to a JSON text, efficiently

    - replacement of any values
    - prepend/insert/replace/append to arrays
    - addition/removal of names from objects
    - setting null values to object names

   no matter how deeply (subject to a max) or shallowly nested these
   values be in the original, while being easy to produce and apply.
Note: it helps to use If-Match to make sure that the resource being
PATCHed has not been modified since it was fetched.  This allows one to
express edits with confidence.

I have two proposals: one based on augmenting a subset of the original
JSON text, the other based a sequence of paths and edit instructions for
the values found at those paths.

Proposal #1: Patch as augmented subset of the original

Follow these steps to produce a patch:

 - Start with the resource's JSON representation.

 - Remove all interior object name/value pairs that are not on the way
   to any value to be added/replaced/removed/otherwise edited.

 - Remove all interior array elements that are not on the way to any
   value to be added/replaced/removed UNLESS the element is in an array
   to be edited (more on this below).

 - Insert before each remaining array element its index.

 - To replace any value with a scalar, just write the new value where
   the old one appeared.

 - To append to any array, just append -1 and the new value to it (after
   the above removals and additions).

 - For any array you want to do more to than append -> wrap the array in
   an object with any of these named values:

    o "delete":  [<list of array indices>]
    o "prepend": [<list of values to append>]
    o "insert":  [<list of [<index>, <value>]>]
    o "set":     [<list of [<index>, <value>]>]
    o "new":     [<list of new elements to replace the old one's with>]
    o "edits":   [<list of edits (see below)>]

   To disambiguate these objects from objects that could legitimately
   appear in the resource the following named value MUST also be

    o "<URN TBD>": true

   Replacements to be performed first, then deletions, then insertions,
   then prepends, then "edits".  Edits are what would have appeared
   had there been no need to wrap the array in order to do delete,
   prepend, replace, or insert elements.

 - To add name/value pairs to objects, just add them.

 - If you want to delete values from objects -> wrap the object in an
   object with any of these names:

    o "delete": [<list of name strings>]
    o "edits":  <object containint edits>

   Edits are what would have appeared had there been no need to wrap the
   object to delete name/value pairs.

   To disambiguate these names the following named value MUST also be

    o "<URN TBD>": true

Values to be added/set appear as they should appear in the resource as

Patch application is straightforward, obvious.

Example resource:

    "a": [ { "b": 1, "c": 2 }, { "d": { "e": [ 2, {} ] } } ],
    "z": [ true, false, null, null, "hello" ]

Example edits:

 - Delete that empty object (path a[0].d.e[1] in jq-speak)

   { "a": [ 1, { "d": { "e": { "delete": [ 1 ],
                               "": true } } } ] }

 - Replace the array [ 2, {} ] (path a[0].d.e in jq-speak) with the
   value 0:

   { "a": [ 1, { "d": { "e": 0 } } ] }

 - Append true to the array [ 2, {} ] (path a[0].d.e in jq-speak):

   { "a": [ 1, { "d": { "e": [ true ] } } ] }

 - Delete the 1, prepend "foo", append "bar" to that same array:

   { "a": [ 1, { "d": { "e": { "delete": [ 0 ],
                               "prepend": [ "foo" ],
                               "edits": [ "bar" ],
                               "": true } } } ] }

 - Add a sibling name to "b" and "c":

   { "a": [ 0, { "f": "hello" } ] }

 - Delete "b":

   { "a": [ 0, { "delete": [ "b" ], "": true } ] }

 - Replace the "z" array with [1, 2, 3]:

   { "z": { "new": [ 1, 2, 3, ], "": true } }

Proposal #2: Use sequence of [<path>, <edit instruction>]

 - Write the path to each value to be edited as an array of path
   elements (strings for object names, numbers for array elements).

 - To replace any value with a scalar value (non-array, non-object),
   just write the path to the value and the new value as the

 - To add a value to an object just write the path to the object, append
   the new value's name to the path, and the instruction will be the new

 - To add (append) a value to an array, write the path to the array,
   append a -1 to the path, and the instruction will be the new value.

 - Edit instructions for arrays (other than appending to, or replacing
   the whole array with a scalar value) are any objects like:

    o "delete":  [<list of indices>] or true (if path names an array

    o "add":     [<list of <value>s to append]

    o "insert":  [<list of [<index>, <value>] pairs>]
    o "set":     [<list of [<index>, <value>] pairs>]
    o "prepend": [<list of <value>s to prepend]
    o "add":     [<list of <value>s to append]
    o "new":     [<new list of values to replace old ones>]

   No magic URN name is needed in this case.

 - To add an object

 - Edit instructions for objects (other than replacement with a scalar
   value) are objects with name/value pairs like:

    o "delete": [<list of name strings>]
    o "merge":  [<object whose named value pairs will replace the
                 corresponding ones of the object being edited>]

   No magic URN name is needed in this case.

The totality of the patch, then, is an array of
[ <path>, <instruction> ] elements.

Any given path could be referenced multiple times in one patch.

Proposal #2 is pretty self-explanatory.  Examples left as an exercise
for the reader.


json mailing list