SWI-Prolog -- Supporting JSON

Documentation
- Reference manual
- Packages
  - SWI-Prolog HTTP support

8 Supporting JSON

From http://json.org, " JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language, Standard ECMA-262 3rd Edition - December 1999. JSON is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages, including C, C++, C#, Java, JavaScript, Perl, Python, and many others. These properties make JSON an ideal data-interchange language."

Although JSON is nowadays used a lot outside the context of web applications, SWI-Prolog's support for JSON started life as part of the HTTP package. SWI-Prolog supports two Prolog representations for JSON terms. The first and oldest map JSON objects to a term json(PropertyList) and use the @ functor to disambiguate e.g. null from the string "null", leading to @(null). As of SWI-Prolog version 7, JSON objects may be represented using dict objects and JSON strings using Prolog strings. Predicates following this convention are suffixed with _dict, e.g. json_read_dict/2. For example, given the JSON document

{ "name": "Bob", "children": ["Mary", "John"], "age":42, "married": true }

we get either (using json_read/2):

json([name='Bob', children=['Mary', 'John'], age=42, married= @(true)]).

or (using json_read_dict/2):

_{age:42, children:["Mary", "John"], married:true, name:"Bob"}

The SWI-Prolog JSON interface consists of three libraries:

library(http/json) provides support for the core JSON object serialization and parsing.
library(http/json_convert) converts between the primary representation of JSON terms in Prolog and more application oriented Prolog terms. E.g. point(X,Y) vs. object([x=X,y=Y]).
library(http/http_json) hooks the conversion libraries into the HTTP client and server libraries.

8.1 library(http/json): Reading and writing JSON serialization

author
Jan Wielemaker

See also
- http_json.pl links JSON to the HTTP client and server modules.
- json_convert.pl converts JSON Prolog terms to more comfortable terms.

This module supports reading and writing JSON objects. This library supports two Prolog representations (the new representation is only supported in SWI-Prolog version 7 and later):
- The classical representation is provided by json_read/3 and json_write/3. This represents a JSON object as json(NameValueList), a JSON string as an atom and the JSON constants null, true and false as @(null), @(true) and @false.
- The new representation is provided by json_read_dict/3 and json_write_dict/3. This represents a JSON object as a dict, a JSON string as a Prolog string and the JSON constants using the Prolog atoms null, true and false.
[det]atom_json_term(?Atom, ?JSONTerm, +Options)
Convert between textual representation and a JSON term. In write mode (JSONTerm to Atom), the option

as(Type)
defines the output type, which is one of atom (default), string, codes or chars.

[det]json_read(+Stream, -Term)

[det]json_read(+Stream, -Term, +Options)
Read next JSON value from Stream into a Prolog term. The canonical representation for Term is:
- A JSON object is mapped to a term json(NameValueList), where NameValueList is a list of Name=Value. Name is an atom created from the JSON string.
- A JSON array is mapped to a Prolog list of JSON values.
- A JSON string is mapped to a Prolog atom
- A JSON number is mapped to a Prolog number
- The JSON constants true and false are mapped -like JPL- to @(true) and @(false).
- The JSON constant null is mapped to the Prolog term @(null)
Here is a complete example in JSON and its corresponding Prolog term.
```
{ "name":"Demo term",
  "created": {
    "day":null,
    "month":"December",
    "year":2007
  },
  "confirmed":true,
  "members":[1,2,3]
}
```
```
json([ name='Demo term',
       created=json([day= @null, month='December', year=2007]),
       confirmed= @true,
       members=[1, 2, 3]
     ])
```
The following options are processed:
null(+NullTerm)
Term used to represent JSON null. Default @(null)

true(+TrueTerm)
Term used to represent JSON true. Default @(true)

false(+FalseTerm)
Term used to represent JSON false. Default @(false)

end_of_file(+ErrorOrTerm)
If end of file is reached after skipping white space but before any input is processed take the following action (default error):

If ErrorOrTerm == error, throw an unexpected end of file syntax error

Otherwise return ErrorOrTerm.

Returning an status term is required to process Concatenated JSON. Suggested values are @(eof) or end_of_file.
value_string_as(+Type)
Prolog type used for strings used as value. Default is atom. The alternative is string, producing a packed string object. Please note that codes or chars would produce ambiguous output and are therefore not supported.
See also
json_read_dict/3 to read a JSON term using the version 7 extended data types.
[det]json_write(+Stream, +Term)

[det]json_write(+Stream, +Term, +Options)
Write a JSON term to Stream. The JSON object is of the same format as produced by json_read/2, though we allow for some more flexibility with regard to pairs in objects. All of Name=Value, Name-Value and Name(Value) produce the same output.
Values can be of the form #(Term), which causes Term to be stringified if it is not an atom or string. Stringification is based on term_string/2.

Rational numbers are emitted as floating point numbers. The hook json_write_hook/4 can be used to realize domain specific alternatives.

The version 7 dict type is supported as well. Optionally, if the dict has a tag, a property "type":"tag" can be added to the object. This behaviour can be controlled using the tag option (see below). For example:
```
?- json_write(current_output, point{x:1,y:2}).
{
  "x":1,
  "y":2
}
```
```
?- json_write(current_output, point{x:1,y:2}, [tag(type)]).
{
  "type":"point",
  "x":1,
  "y":2
}
```
In addition to the options recognised by json_read/3, we process the following options are recognised:

width(+Width)
Width in which we try to format the result. Too long lines switch from horizontal to vertical layout for better readability. If performance is critical and human readability is not an issue use Width = 0, which causes a single-line output.

step(+Step)
Indentation increnment for next level. Default is 2.

tab(+TabDistance)
Distance between tab-stops. If equal to Step, layout is generated with one tab per level.

serialize_unknown(+Boolean)
If true (default false), serialize unknown terms and print them as a JSON string. The default raises a type error. Note that this option only makes sense if you can guarantee that the passed value is not an otherwise valid Prolog reporesentation of a Prolog term.

If a string is emitted, the sequence </ is emitted as <\/. This is valid JSON syntax which ensures that JSON objects can be safely embedded into an HTML <script> element.
[semidet,multifile]json_write_hook(+Term, +Stream, +State, +Options)
Hook that can be used to emit a JSON representation for Term to Stream. If the predicate succeeds it must have written a valid JSON data element and if it fails it may not have produced any output. This facility may be used to map arbitrary Prolog terms to JSON. It was added to manage the precision with which floating point numbers are emitted.
Note that this hook is shared by all users of this library. It is generally adviced to map a unique compound term to avoid interference with normal output.

State and Options are opaque handles to the current output state and settings. Future versions may provide documented access to these terms. Currently it is adviced to ignore these arguments.

[semidet,multifile]json_dict_pairs(+Dict, -Pairs)
This hook may be used to order the keys of an object. If it fails, dict_pairs/3 is used which produces an ordered list of keys.

[semidet]is_json_term(@Term)

[semidet]is_json_term(@Term, +Options)
True if Term is a json term. Options are the same as for json_read/2, defining the Prolog representation for the JSON true, false and null constants.

[det]json_read_dict(+Stream, -Dict)

[det]json_read_dict(+Stream, -Dict, +Options)
Read a JSON object, returning objects as a dicts. The representation depends on the options, where the default is:
- String values are mapped to Prolog strings
- JSON true, false and null are represented using these Prolog atoms.
- JSON objects are mapped to dicts.
- Optionally, a type field in an object assigns a tag for the dict.
The predicate json_read_dict/3 processes the same options as json_read/3, but with different defaults. In addition, it processes the tag option. See json_read/3 for details about the shared options.

tag(+Name)
When converting to/from a dict, map the indicated JSON attribute to the dict tag. No mapping is performed if Name is the empty atom (” , default). See json_read_dict/2 and json_write_dict/2.

default_tag(+Tag)
Provide the default tag if the above tag option does not apply.

null(+NullTerm)
Default the atom null.

true(+TrueTerm)
Default the atom true.

false(+FalseTerm)
Default the atom false

end_of_file(+ErrorOrTerm)
Action on reading end-of-file. See json_read/3 for details.

value_string_as(+Type)
Prolog type used for strings used as value. Default is string. The alternative is atom, producing a packed string object.
[det]json_write_dict(+Stream, +Dict)

[det]json_write_dict(+Stream, +Dict, +Options)
Write a JSON term, represented using dicts. This is the same as json_write/3, but assuming the default representation of JSON objects as dicts.

[det]atom_json_dict(+Atom, -JSONDict, +Options)

[det]atom_json_dict(-Text, +JSONDict, +Options)
Convert between textual representation and a JSON term represented as a dict. Options are as for json_read/3. In write mode, the addtional option

as(Type)
defines the output type, which is one of atom, string or codes.
8.2 library(http/json_convert): Convert between JSON terms and Prolog application terms
To be done
- Ignore extra fields. Using a partial list of extra?
- Consider a sensible default for handling JSON null. Conversion to Prolog could translate @null into a variable if the desired type is not any. Conversion to JSON could map variables to null, though this may be unsafe. If the Prolog term is known to be non-ground and JSON @null is a sensible mapping, we can also use this simple snipit to deal with that fact.
```
        term_variables(Term, Vars),
        maplist(=(@null), Vars).
```
The idea behind this module is to provide a flexible high-level mapping between Prolog terms as you would like to see them in your application and the standard representation of a JSON object as a Prolog term. For example, an X-Y point may be represented in JSON as {"x":25, "y":50}. Represented in Prolog this becomes json([x=25,y=50]), but this is a pretty non-natural representation from the Prolog point of view.

This module allows for defining records (just like library(record)) that provide transparent two-way transformation between the two representations.
```
:- json_object
        point(x:integer, y:integer).
```
This declaration causes prolog_to_json/2 to translate the native Prolog representation into a JSON Term:
```
?- prolog_to_json(point(25,50), X).

X = json([x=25, y=50])
```
A json_object/1 declaration can define multiple objects separated by a comma (,), similar to the dynamic/1 directive. Optionally, a declaration can be qualified using a module. The conversion predicates prolog_to_json/2 and json_to_prolog/2 first try a conversion associated with the calling module. If not successful, they try conversions associated with the module user.

JSON objects have no type. This can be solved by adding an extra field to the JSON object, e.g. {"type":"point", "x":25, "y":50}. As Prolog records are typed by their functor we need some notation to handle this gracefully. This is achieved by adding +Fields to the declaration. I.e.
```
:- json_object
        point(x:integer, y:integer) + [type=point].
```
Using this declaration, the conversion becomes:
```
?- prolog_to_json(point(25,50), X).

X = json([x=25, y=50, type=point])
```
The predicate json_to_prolog/2 is often used after http_read_json/2 and prolog_to_json/2 before reply_json/1. For now we consider them separate predicates because the transformation may be too general, too slow or not needed for dedicated applications. Using a separate step also simplifies debugging this rather complicated process.
[multifile]current_json_object(Term, Module, Fields)
Multifile predicate computed from the json_object/1 declarations. Term is the most general Prolog term representing the object. Module is the module in which the object is defined and Fields is a list of f(Name, Type, Default, Var), ordered by Name. Var is the corresponding variable in Term.

json_object(+Declaration)
Declare a JSON object. The declaration takes the same format as using in record/1 from library(record). E.g.
```
?- json_object
      point(x:int, y:int, z:int=0).
```
The type arguments are either types as know to library(error) or functor names of other JSON objects. The constant any indicates an untyped argument. If this is a JSON term, it becomes subject to json_to_prolog/2. I.e., using the type list(any) causes the conversion to be executed on each element of the list.

If a field has a default, the default is used if the field is not specified in the JSON object. Extending the record type definition, types can be of the form (Type1|Type2). The type null means that the field may not be present.

Conversion of JSON to Prolog applies if all non-defaulted arguments can be found in the JSON object. If multiple rules match, the term with the highest arity gets preference.
[semidet]prolog_bool_to_json(+Prolog, -JSON)
JSON is the JSON boolean for Prolog. It is a flexible the Prolog notation for thruth-value, accepting one of true, on or 1 for @true and one of false, fail, off or 0 for @false.

Errors
instantiation_error if Prolog is unbound.

[det]prolog_to_json(:Term, -JSONObject)
Translate a Prolog application Term into a JSON object term. This transformation is based on :- json_object/1 declarations. If a json_object/1 declaration declares a field of type boolean, commonly used thruth-values in Prolog are converted to JSON booleans. Boolean translation accepts one of true, on, 1, @true, false, fail, off or 0, @false.

Errors
- type_error(json_term, X)
- instantiation_error

[det]json_to_prolog(+JSON, -Term)
Translate a JSON term into an application term. This transformation is based on :- json_object/1 declarations. An efficient transformation is non-trivial, but we rely on the assumption that, although the order of fields in JSON terms is irrelevant and can therefore vary a lot, practical applications will normally generate the JSON objects in a consistent order.
If a field in a json_object is declared of type boolean, @true and @false are translated to true or false, the most commonly used Prolog representation for truth-values.
8.3 library(http/http_json): HTTP JSON Plugin module

See also
- JSON Requests are discussed in http://json.org/JSONRequest.html
- json.pl describes how JSON objects are represented in Prolog terms.
- json_convert.pl converts between more natural Prolog terms and json terms.

Most code doesn't need to use this directly; instead use library(http/http_server), which combines this library with the typical HTTP libraries that most servers need.

This module adds hooks to several parts of the HTTP libraries, making them JSON-aware. Notably:
- Make http_read_data/3 convert application/json and application/jsonrequest content to a JSON term.
- Cause http_open/3 to accept post(json(Term)) to issue a POST request with JSON content.
- Provide HTTP server and client utility predicates for reading and replying JSON:
- Reply to exceptions in the server using an JSON document rather then HTML if the Accept header prefers application/json over text/html.
Typically JSON is used by Prolog HTTP servers. This module supports two JSON representations: the classical representation and the new representation supported by the SWI-Prolog version 7 extended data types. Below is a skeleton for handling a JSON request, answering in JSON using the classical interface.
```
handle(Request) :-
      http_read_json(Request, JSONIn),
      json_to_prolog(JSONIn, PrologIn),
      <compute>(PrologIn, PrologOut),         % application body
      prolog_to_json(PrologOut, JSONOut),
      reply_json(JSONOut).
```
When using dicts, the conversion step is generally not needed and the code becomes:
```
handle(Request) :-
      http_read_json_dict(Request, DictIn),
      <compute>(DictIn, DictOut),
      reply_json(DictOut).
```
This module also integrates JSON support into the http client provided by http_client.pl. Posting a JSON query and processing the JSON reply (or any other reply understood by http_read_data/3) is as simple as below, where Term is a JSON term as described in json.pl and reply is of the same format if the server replies with JSON.
```
      ...,
      http_post(URL, json(Term), Reply, [])
```
[multifile]http_client:http_convert_data(+In, +Fields, -Data, +Options)
Hook implementation that supports reading JSON documents. It processes the following option:

json_object(+As)
Where As is one of term or dict. If the value is dict, json_read_dict/3 is used.

[semidet]is_json_content_type(+ContentType)
True if ContentType is a header value (either parsed or as atom/string) that denotes a JSON value.

[semidet,multifile]json_type(?MediaType)
True if MediaType is a JSON media type. http_json:json_type/1 is a multifile predicate and may be extended to facilitate non-conforming clients.

MediaType is a term Type/SubType, where both Type and SubType are atoms.

[semidet,multifile]http:post_data_hook(+Data, +Out:stream, +HdrExtra)
Hook implementation that allows http_post_data/3 posting JSON objects using one of the forms below.
```
http_post(URL, json(Term), Reply, Options)
http_post(URL, json(Term, Options), Reply, Options)
```
If Options are passed, these are handed to json_write/3. In addition, this option is processed:

json_object(As)
If As is dict, json_write_dict/3 is used to write the output. This is default if json(Dict) is passed.

To be done
avoid creation of intermediate data using chunked output.
[det]http_read_json(+Request, -JSON)

[det]http_read_json(+Request, -JSON, +Options)
Extract JSON data posted to this HTTP request. Options are passed to json_read/3. In addition, this option is processed:

json_object(+As)
One of term (default) to generate a classical Prolog term or dict to exploit the SWI-Prolog version 7 data type extensions. See json_read_dict/3.

Errors
- domain_error(mimetype, Found) if the mimetype is not known (see json_type/1).
- domain_error(method, Method) if the request method is not a POST, PUT or PATCH.

[det]http_read_json_dict(+Request, -Dict)

[det]http_read_json_dict(+Request, -Dict, +Options)
Similar to http_read_json/2,3, but by default uses the version 7 extended datatypes.

[det]reply_json(+JSONTerm)

[det]reply_json(+JSONTerm, +Options)
Formulate a JSON HTTP reply. See json_write/2 for details. The processed options are listed below. Remaining options are forwarded to json_write/3.

content_type(+Type)
The default Content-type is application/json; charset=UTF8. charset=UTF8 should not be required because JSON is defined to be UTF-8 encoded, but some clients insist on it.

status(+Code)
The default status is 200. REST API functions may use other values from the 2XX range, such as 201 (created).

json_object(+As)
One of term (classical json representation) or dict to use the new dict representation. If omitted and Term is a dict, dict is assumed. SWI-Prolog Version 7.

[det]reply_json_dict(+JSONTerm)

[det]reply_json_dict(+JSONTerm, +Options)
As reply_json/1 and reply_json/2, but assumes the new dict based data representation. Note that this is the default if the outer object is a dict. This predicate is needed to serialize a list of objects correctly and provides consistency with http_read_json_dict/2 and friends.

8 Supporting JSON

8.1 library(http/json): Reading and writing JSON serialization

8.2 library(http/json_convert): Convert between JSON terms and Prolog application terms

8.3 library(http/http_json): HTTP JSON Plugin module