JSON Diff Explained

Text diff and semantic diff

The usual diff tools, such as the diff command and git diff, compare text line by line. They do not know what JSON is. Two files with identical data can have many differences as text, because of indentation, line breaks or the order of keys. A semantic JSON diff parses both documents and compares the data, and ignores everything that is only formatting.

How to read a path

Each difference has a path that shows where it is. The dollar sign is the root, a dot goes into an object key and square brackets pick an array item.

$.orders[2].items[0].price

That is the price of the first item of the third order. When arrays are matched by a key, the position is replaced by the key, as in $.users[id=7].email.

The kinds of change

• Added, removed and changed values.
• Type changes, where the value moved from one type to another, for example the number 5 to the text "5", or a value to null.

Arrays are the hard part

An object has keys, so each value has a clear partner. An array has only positions. If a new item is inserted at the start of a list, every other item moves one place, and a comparison by position reports that all of them changed. There are three ways to handle this:

1. By position is right when the order has meaning, such as the steps in a process.
2. Ignoring order is right for sets, such as tags. Items are matched by content.
3. Matching by key is right for lists of records. Items are paired by an id, and each pair is compared in depth.

Things that cause false differences

• Volatile fields such as timestamps, request ids and tokens. Ignore them by name.
• Number formatting. 1.0, 1 and 1e0 are the same number in JSON. Numbers beyond about 15 digits are rounded by JavaScript, so two different very large ids can compare as equal. Compare those as strings.
• null and a missing key are different, and some APIs switch between them.
• Dates written in different formats for the same moment are different strings.
• Duplicate keys in one object are legal to write, but most parsers keep only the last value, so the first one is lost before the comparison.
• Floating point results such as 0.1 plus 0.2 may differ in the last digits.

Comparing in the terminal

With jq you can make a text diff behave like a semantic one by sorting the keys first:

diff <(jq -S . a.json) <(jq -S . b.json)

The -S option sorts the keys and the output is indented in a standard way. This still compares arrays by position.

Patches

A diff can also be written as instructions for turning one document into the other. JSON Patch (RFC 6902) is a list of operations such as add, remove and replace, each with a path written as a JSON Pointer. JSON Merge Patch (RFC 7396) is a smaller document that is laid over the original. APIs use them to send only what changed.

Where JSON diff helps

• Checking an API response after a code change.
• Reviewing configuration changes between environments.
• Test snapshots, where the expected and actual output are compared.
• Finding out what changed between two exports of the same data.

Try it with the JSON diff tool, use JSON compare for the options, or see the changes in the JSON diff viewer.