Roll //thirdparty/pyjson5 to v0.9.5.

The current version of pyjson5 was v0.6.1, more than two years old :(.

There's no real significant new functionality in this new version and
the performance is still slow, but there are a number of bug fixes.

R=aboxhall@chromium.org

Change-Id: Ic293163382a1b0fa1111aca5b04a455eb9b72a45
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2216513
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#776381}

third_party/blink/renderer/bindings/scripts/build_web_idl_database.pydeps

 # Generated by running:
 #   build/print_python_deps.py --root third_party/blink/renderer/bindings/scripts --output third_party/blink/renderer/bindings/scripts/build_web_idl_database.pydeps third_party/blink/renderer/bindings/scripts/build_web_idl_database.py
 ../../../../pyjson5/src/json5/__init__.py
-../../../../pyjson5/src/json5/arg_parser.py
-../../../../pyjson5/src/json5/host.py
 ../../../../pyjson5/src/json5/lib.py
 ../../../../pyjson5/src/json5/parser.py
-../../../../pyjson5/src/json5/tool.py
 ../../../../pyjson5/src/json5/version.py
 ../../build/scripts/blinkbuild/__init__.py
 ../../build/scripts/blinkbuild/name_style_converter.py

third_party/blink/renderer/bindings/scripts/collect_idl_files.pydeps

@@ -8,11 +8,8 @@
 ../../../../ply/lex.py
 ../../../../ply/yacc.py
 ../../../../pyjson5/src/json5/__init__.py
-../../../../pyjson5/src/json5/arg_parser.py
-../../../../pyjson5/src/json5/host.py
 ../../../../pyjson5/src/json5/lib.py
 ../../../../pyjson5/src/json5/parser.py
-../../../../pyjson5/src/json5/tool.py
 ../../../../pyjson5/src/json5/version.py
 ../../build/scripts/blinkbuild/__init__.py
 ../../build/scripts/blinkbuild/name_style_converter.py

third_party/blink/renderer/bindings/scripts/generate_bindings.pydeps

@@ -20,11 +20,8 @@
 ../../../../markupsafe/_compat.py
 ../../../../markupsafe/_native.py
 ../../../../pyjson5/src/json5/__init__.py
-../../../../pyjson5/src/json5/arg_parser.py
-../../../../pyjson5/src/json5/host.py
 ../../../../pyjson5/src/json5/lib.py
 ../../../../pyjson5/src/json5/parser.py
-../../../../pyjson5/src/json5/tool.py
 ../../../../pyjson5/src/json5/version.py
 ../../build/scripts/blinkbuild/__init__.py
 ../../build/scripts/blinkbuild/name_style_converter.py

third_party/blink/renderer/bindings/scripts/generate_high_entropy_list.pydeps

 # Generated by running:
 #   build/print_python_deps.py --root third_party/blink/renderer/bindings/scripts --output third_party/blink/renderer/bindings/scripts/generate_high_entropy_list.pydeps third_party/blink/renderer/bindings/scripts/generate_high_entropy_list.py
 ../../../../pyjson5/src/json5/__init__.py
-../../../../pyjson5/src/json5/arg_parser.py
-../../../../pyjson5/src/json5/host.py
 ../../../../pyjson5/src/json5/lib.py
 ../../../../pyjson5/src/json5/parser.py
-../../../../pyjson5/src/json5/tool.py
 ../../../../pyjson5/src/json5/version.py
 ../../build/scripts/blinkbuild/__init__.py
 ../../build/scripts/blinkbuild/name_style_converter.py

third_party/pyjson5/README.chromium

 Name: pyjson5
 Short Name: pyjson5
 URL: https://github.com/dpranke/pyjson5
-Version: 0.6.1
-Date: May 22, 2018
-Revision: c88ac1f
+Version: 0.9.5
+Date: May 26 2020
+Revision: 9335da8
 License: Apache 2.0
 License File: src/LICENSE
 Security Critical: No

third_party/pyjson5/src/MANIFEST.in

+include *.md
+include LICENSE

third_party/pyjson5/src/README.md

+# pyjson5
+
+A Python implementation of the JSON5 data format.
+
+[JSON5](https://json5.org) extends the
+[JSON](http://www.json.org) data interchange format to make it
+slightly more usable as a configuration language:
+
+* JavaScript-style comments (both single and multi-line) are legal.
+
+* Object keys may be unquoted if they are legal ECMAScript identifiers
+
+* Objects and arrays may end with trailing commas.
+
+* Strings can be single-quoted, and multi-line string literals are allowed.
+
+There are a few other more minor extensions to JSON; see the above page for
+the full details.
+
+This project implements a reader and writer implementation for Python;
+where possible, it mirrors the
+[standard Python JSON API](https://docs.python.org/library/json.html)
+package for ease of use.
+
+There is one notable difference from the JSON api: the `load()` and
+`loads()` methods support optionally checking for (and rejecting) duplicate
+object keys; pass `allow_duplicate_keys=False` to do so (duplicates are
+allowed by default).
+
+This is an early release. It has been reasonably well-tested, but it is
+**SLOW**. It can be 1000-6000x slower than the C-optimized JSON module,
+and is 200x slower (or more) than the pure Python JSON module.
+
+## Known issues
+
+* Did I mention that it is **SLOW**?
+
+* The implementation follows Python3's `json` implementation where
+  possible. This means that the `encoding` method to `dump()` is
+  ignored, and unicode strings are always returned.
+
+* The `cls` keyword argument that `json.load()`/`json.loads()` accepts
+  to specify a custom subclass of ``JSONDecoder`` is not and will not be
+  supported, because this implementation uses a completely different
+  approach to parsing strings and doesn't have anything like the
+  `JSONDecoder` class.
+
+* The `cls` keyword argument that `json.dump()`/`json.dumps()` accepts
+  is also not supported, for consistency with `json5.load()`. The `default`
+  keyword *is* supported, though, and might be able to serve as a
+  workaround.
+
+## Running the tests
+To run the tests, setup a venv and install the required dependencies with
+`pip install -e '.[dev]'`, then run the tests with `python setup.py test`.
+
+
+## Version History / Release Notes
+
+* v0.9.5 (2020-05-26)
+    * Miscellaneous non-source cleanups in the repo, including setting
+      up GitHub Actions for a CI system. No changes to the library from
+      v0.9.4, other than updating the version.
+
+* v0.9.4 (2020-03-26)
+    * [GitHub pull #38](https://github.com/dpranke/pyjson5/pull/38)
+      Fix from fredrik@fornwall.net for dumps() crashing when passed
+      an empty string as a key in an object.
+
+* v0.9.3 (2020-03-17)
+    * [GitHub pull #35](https://github.com/dpranke/pyjson5/pull/35)
+      Fix from pastelmind@ for dump() not passing the right args to dumps().
+    * Fix from p.skouzos@novafutur.com to remove the tests directory from
+      the setup call, making the package a bit smaller.
+
+* v0.9.2 (2020-03-02)
+    * [GitHub pull #34](https://github.com/dpranke/pyjson5/pull/34)
+      Fix from roosephu@ for a badly formatted nested list.
+
+* v0.9.1 (2020-02-09)
+    * [GitHub issue #33](https://github.com/dpranke/pyjson5/issues/33):
+       Fix stray trailing comma when dumping an object with an invalid key.
+
+* v0.9.0 (2020-01-30)
+    * [GitHub issue #29](https://github.com/dpranke/pyjson5/issues/29):
+       Fix an issue where objects keys that started with a reserved
+       word were incorrectly quoted.
+    * [GitHub issue #30](https://github.com/dpranke/pyjson5/issues/30):
+       Fix an issue where dumps() incorrectly thought a data structure
+       was cyclic in some cases.
+    * [GitHub issue #32](https://github.com/dpranke/pyjson5/issues/32):
+       Allow for non-string keys in dicts passed to ``dump()``/``dumps()``.
+       Add an ``allow_duplicate_keys=False`` to prevent possible
+       ill-formed JSON that might result.
+
+* v0.8.5 (2019-07-04)
+    * [GitHub issue #25](https://github.com/dpranke/pyjson5/issues/25):
+      Add LICENSE and README.md to the dist.
+    * [GitHub issue #26](https://github.com/dpranke/pyjson5/issues/26):
+      Fix printing of empty arrays and objects with indentation, fix
+      misreporting of the position on parse failures in some cases.
+
+* v0.8.4 (2019-06-11)
+    * Updated the version history, too.
+
+* v0.8.3 (2019-06-11)
+    * Tweaked the README, bumped the version, forgot to update the version
+      history :).
+
+* v0.8.2 (2019-06-11)
+    * Actually bump the version properly, to 0.8.2.
+
+* v0.8.1 (2019-06-11)
+    * Fix bug in setup.py that messed up the description. Unfortunately,
+      I forgot to bump the version for this, so this also identifies as 0.8.0.
+
+* v0.8.0 (2019-06-11)
+    * Add `allow_duplicate_keys=True` as a default argument to
+      `json5.load()`/`json5.loads()`. If you set the key to `False`, duplicate
+      keys in a single dict will be rejected. The default is set to `True`
+      for compatibility with `json.load()`, earlier versions of json5, and
+      because it's simply not clear if people would want duplicate checking
+      enabled by default.
+
+* v0.7 (2019-03-31)
+    * Changes dump()/dumps() to not quote object keys by default if they are
+      legal identifiers. Passing `quote_keys=True` will turn that off
+      and always quote object keys.
+    * Changes dump()/dumps() to insert trailing commas after the last item
+      in an array or an object if the object is printed across multiple lines
+      (i.e., if `indent` is not None). Passing `trailing_commas=False` will
+      turn that off.
+    * The `json5.tool` command line tool now supports the `--indent`,
+      `--[no-]quote-keys`, and `--[no-]trailing-commas` flags to allow
+      for more control over the output, in addition to the existing
+      `--as-json` flag.
+    * The `json5.tool` command line tool no longer supports reading from
+      multiple files, you can now only read from a single file or
+      from standard input.
+    * The implementation no longer relies on the standard `json` module
+      for anything. The output should still match the json module (except
+      as noted above) and discrepancies should be reported as bugs.
+
+* v0.6.2 (2019-03-08)
+    * Fix [GitHub issue #23](https://github.com/dpranke/pyjson5/issues/23) and
+      pass through unrecognized escape sequences.
+
+* v0.6.1 (2018-05-22)
+    * Cleaned up a couple minor nits in the package.
+
+* v0.6.0 (2017-11-28)
+    * First implementation that attempted to implement 100% of the spec.
+
+* v0.5.0 (2017-09-04)
+    * First implementation that supported the full set of kwargs that
+      the `json` module supports.

third_party/pyjson5/src/benchmarks/chromium.linux.json

@@ -4133,15 +4133,15 @@
         }
       },
       {
-        "isolate_name": "devtools_lint_check",
-        "name": "devtools_lint_check",
+        "isolate_name": "devtools_closure_compile",
+        "name": "devtools_closure_compile",
         "swarming": {
           "can_use_on_swarming_builders": true
         }
       },
       {
-        "isolate_name": "devtools_type_check",
-        "name": "devtools_type_check",
+        "isolate_name": "devtools_eslint",
+        "name": "devtools_eslint",
         "swarming": {
           "can_use_on_swarming_builders": true
         }

third_party/pyjson5/src/benchmarks/mb_config.json

@@ -1966,7 +1966,7 @@
       "gn_args": "internal_gles2_conform_tests=true"
     }, 
     "java_coverage": {
-      "gn_args": "jacoco_coverage=true"
+      "gn_args": "emma_coverage=true emma_filter=\"org.chromium.*\""
     }, 
     "libfuzzer": {
       "gn_args": "use_libfuzzer=true"

third_party/pyjson5/src/benchmarks/run.py

@@ -22,11 +22,12 @@ import sys
 import time
 
 THIS_DIR = os.path.abspath(os.path.dirname(__file__))
-REPO_DIR  = os.path.dirname(THIS_DIR)
+REPO_DIR = os.path.dirname(THIS_DIR)
 if not REPO_DIR in sys.path:
     sys.path.insert(0, REPO_DIR)
 
-import json5
+import json5  # pylint: disable=wrong-import-position
+
 
 ALL_BENCHMARKS = (
     'ios-simulator.json',
@@ -35,7 +36,9 @@ ALL_BENCHMARKS = (
     'chromium.perf.json',
 )
 
-DEFAULT_ITERATIONS = 1
+
+DEFAULT_ITERATIONS = 3
+
 
 def main():
     parser = argparse.ArgumentParser()
@@ -54,18 +57,21 @@ def main():
 
 
     # json.decoder.c_scanstring = py_scanstring
-    def py_maker(*_args, **_kwargs):
-      decoder = json.JSONDecoder()
-      decoder.scan_once = json.scanner.py_make_scanner(decoder)
-      decoder.parse_string = json.decoder.py_scanstring
-      json.decoder.scanstring = decoder.parse_string
-      return decoder
+    def py_maker(*args, **kwargs):
+        del args
+        del kwargs
+        decoder = json.JSONDecoder()
+        decoder.scan_once = json.scanner.py_make_scanner(decoder)
+        decoder.parse_string = json.decoder.py_scanstring
+        json.decoder.scanstring = decoder.parse_string
+        return decoder
 
     maker = py_maker if args.pure else json.JSONDecoder
 
     all_times = []
     for i, c in enumerate(file_contents):
-        times = []
+        json_time = 0.0
+        json5_time = 0.0
         for _ in range(args.num_iterations):
             start = time.time()
             json_obj = json.loads(c, cls=maker)
@@ -73,16 +79,30 @@ def main():
             json5_obj = json5.loads(c)
             end = time.time()
 
-            json_time = mid - start
-            json5_time = end - mid
-            times.append((json_time, json5_time))
-            assert(json5_obj == json_obj)
-        all_times.append(times)
-
-    for i, times in enumerate(all_times):
-        avg = sum((json5_time / json_time)
-                  for json_time, json5_time in times) / args.num_iterations
-        print("%-20s: %5.1f" % (args.benchmarks[i], avg))
+            json_time += mid - start
+            json5_time += end - mid
+            assert json5_obj == json_obj
+        all_times.append((json_time, json5_time))
+
+    for i, (json_time, json5_time) in enumerate(all_times):
+        fname = os.path.basename(args.benchmarks[i])
+        if json5_time and json_time:
+            if json5_time > json_time:
+                avg = json5_time / json_time
+                print("%-20s: JSON was %5.1fx faster (%.6fs to %.6fs)" % (
+                      fname, avg, json_time, json5_time))
+            else:
+                avg = json_time / json5_time
+                print("%-20s: JSON5 was %5.1fx faster (%.6fs to %.6fs)" % (
+                      fname, avg, json5_time, json_time))
+        elif json5_time:
+            print("%-20s: JSON5 took %.6f secs, JSON was too fast to measure" %
+                  (fname, json5_time))
+        elif json_time:
+            print("%-20s: JSON took %.6f secs, JSON5 was too fast to measure" %
+                  (fname, json_time))
+        else:
+            print("%-20s: both were too fast to measure" % (fname,))
 
     return 0
 

third_party/pyjson5/src/json5/__init__.py

@@ -14,7 +14,6 @@
 
 """A pure Python implementation of the JSON5 configuration language."""
 
-from . import tool
 from .lib import load, loads, dump, dumps
 from .version import VERSION
 
@@ -25,5 +24,4 @@ __all__ = [
     'dumps',
     'load',
     'loads',
-    'tool',
 ]

third_party/pyjson5/src/json5/arg_parser.py

@@ -22,12 +22,15 @@ class _Bailout(Exception):
 class ArgumentParser(argparse.ArgumentParser):
     SUPPRESS = argparse.SUPPRESS
 
-    def __init__(self, host, **kwargs):
+    def __init__(self, host, prog, desc, **kwargs):
+        kwargs['prog'] = prog
+        kwargs['description'] = desc
+        kwargs['formatter_class'] = argparse.RawDescriptionHelpFormatter
         super(ArgumentParser, self).__init__(**kwargs)
         self._host = host
         self.exit_status = None
         self.add_argument('-V', '--version', action='store_true',
-                          help='Print the version and exit.')
+                          help='print the version and exit')
 
     def parse_args(self, args=None, namespace=None):
         try:

third_party/pyjson5/src/json5/host.py

@@ -12,7 +12,6 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import fileinput
 import os
 import shutil
 import sys
@@ -20,7 +19,7 @@ import tempfile
 
 
 if sys.version_info[0] < 3:
-    # pylint: disable=redefined-builtin
+    # pylint: disable=redefined-builtin, invalid-name
     str = unicode
 
 
@@ -33,11 +32,6 @@ class Host(object):
     def chdir(self, *comps):
         return os.chdir(self.join(*comps))
 
-    def fileinput(self, files=None):
-        if not files:
-            return self.stdin.readlines()
-        return fileinput.input(files)
-
     def getcwd(self):
         return os.getcwd()
 
@@ -55,6 +49,10 @@ class Host(object):
     def rmtree(self, path):
         shutil.rmtree(path, ignore_errors=True)
 
+    def read_text_file(self, path):
+        with open(path, 'rb') as fp:
+            return fp.read().decode('utf8')
+
     def write_text_file(self, path, contents):
         with open(path, 'wb') as f:
             f.write(contents.encode('utf8'))

third_party/pyjson5/src/json5/json5.g

@@ -52,6 +52,8 @@ esc_char       = 'b'                                 -> '\u0008'
                | squote                              -> '\u0027'
                | dquote                              -> '\u0022'
                | bslash                              -> '\u005C'
+               | ~('x'|'u'|digit|eol) anything:c     -> c
+               | '0' ~digit                          -> '\u0000'
                | hex_esc:c                           -> c
                | unicode_esc:c                       -> c
 

third_party/pyjson5/src/json5/lib.py

@@ -12,34 +12,55 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+import math
 import re
-import json
 import sys
+import unicodedata
 
 from .parser import Parser
 
 
 if sys.version_info[0] < 3:
-    # pylint: disable=redefined-builtin
-    str = unicode
+    str = unicode  # pylint: disable=redefined-builtin, invalid-name
+else:
+    long = int  # pylint: disable=redefined-builtin, invalid-name
 
 
 def load(fp, encoding=None, cls=None, object_hook=None, parse_float=None,
-         parse_int=None, parse_constant=None, object_pairs_hook=None):
+         parse_int=None, parse_constant=None, object_pairs_hook=None,
+         allow_duplicate_keys=True):
     """Deserialize ``fp`` (a ``.read()``-supporting file-like object
-    containing a JSON document) to a Python object."""
+    containing a JSON document) to a Python object.
+
+    Supports almost the same arguments as ``json.load()`` except that:
+        - the `cls` keyword is ignored.
+        - an extra `allow_duplicate_keys` parameter supports checking for
+          duplicate keys in a object; by default, this is True for
+          compatibility with ``json.load()``, but if set to False and
+          the object contains duplicate keys, a ValueError will be raised.
+    """
 
     s = fp.read()
     return loads(s, encoding=encoding, cls=cls, object_hook=object_hook,
                  parse_float=parse_float, parse_int=parse_int,
                  parse_constant=parse_constant,
-                 object_pairs_hook=object_pairs_hook)
+                 object_pairs_hook=object_pairs_hook,
+                 allow_duplicate_keys=allow_duplicate_keys)
 
 
 def loads(s, encoding=None, cls=None, object_hook=None, parse_float=None,
-          parse_int=None, parse_constant=None, object_pairs_hook=None):
+          parse_int=None, parse_constant=None, object_pairs_hook=None,
+          allow_duplicate_keys=True):
     """Deserialize ``s`` (a ``str`` or ``unicode`` instance containing a
-    JSON5 document) to a Python object."""
+    JSON5 document) to a Python object.
+
+    Supports the same arguments as ``json.load()`` except that:
+        - the `cls` keyword is ignored.
+        - an extra `allow_duplicate_keys` parameter supports checking for
+          duplicate keys in a object; by default, this is True for
+          compatibility with ``json.load()``, but if set to False and
+          the object contains duplicate keys, a ValueError will be raised.
+    """
 
     assert cls is None, 'Custom decoders are not supported'
 
@@ -54,7 +75,7 @@ def loads(s, encoding=None, cls=None, object_hook=None, parse_float=None,
     if not s:
         raise ValueError('Empty strings are not legal JSON5')
     parser = Parser(s, '<string>')
-    ast, err, newpos = parser.parse()
+    ast, err, _ = parser.parse()
     if err:
         raise ValueError(err)
 
@@ -66,7 +87,11 @@ def loads(s, encoding=None, cls=None, object_hook=None, parse_float=None,
     elif object_hook:
         dictify = lambda pairs: object_hook(dict(pairs))
     else:
-        dictify = dict
+        dictify = lambda pairs: dict(pairs) # pylint: disable=unnecessary-lambda
+
+    if not allow_duplicate_keys:
+        _orig_dictify = dictify
+        dictify = lambda pairs: _reject_duplicate_keys(pairs, _orig_dictify)
 
     parse_float = parse_float or float
     parse_int = parse_int or int
@@ -75,6 +100,14 @@ def loads(s, encoding=None, cls=None, object_hook=None, parse_float=None,
     return _walk_ast(ast, dictify, parse_float, parse_int, parse_constant)
 
 
+def _reject_duplicate_keys(pairs, dictify):
+    keys = set()
+    for key, _ in pairs:
+        if key in keys:
+            raise ValueError('Duplicate key "%s" found in object', key)
+        keys.add(key)
+    return dictify(pairs)
+
 def _walk_ast(el, dictify, parse_float, parse_int, parse_constant):
     if el == 'None':
         return None
@@ -107,53 +140,370 @@ def _walk_ast(el, dictify, parse_float, parse_int, parse_constant):
     raise Exception('unknown el: ' + el)  # pragma: no cover
 
 
-_notletter = re.compile('\W')
+def dump(obj, fp, skipkeys=False, ensure_ascii=True, check_circular=True,
+         allow_nan=True, cls=None, indent=None, separators=None,
+         default=None, sort_keys=False,
+         quote_keys=False, trailing_commas=True,
+         allow_duplicate_keys=True,
+         **kwargs):
+    """Serialize ``obj`` to a JSON5-formatted stream to ``fp`` (a ``.write()``-
+    supporting file-like object).
+
+    Supports the same arguments as ``json.dumps()``, except that:
+
+    - The ``cls`` keyword is not supported.
+    - The ``encoding`` keyword is ignored; Unicode strings are always written.
+    - By default, object keys that are legal identifiers are not quoted;
+      if you pass quote_keys=True, they will be.
+    - By default, if lists and objects span multiple lines of output (i.e.,
+      when ``indent`` >=0), the last item will have a trailing comma
+      after it. If you pass ``trailing_commas=False, it will not.
+    - If you use a number, a boolean, or None as a key value in a dict,
+      it will be converted to the corresponding json string value, e.g.
+      "1", "true", or "null". By default, dump() will match the `json`
+      modules behavior and produce ill-formed JSON if you mix keys of
+      different types that have the same converted value, e.g.:
+      {1: "foo", "1": "bar"} produces '{"1": "foo", "1": "bar"}', an
+      object with duplicated keys. If you pass allow_duplicate_keys=False,
+      an exception will be raised instead.
+
+    Calling ``dumps(obj, fp, quote_keys=True, trailing_commas=False,
+                    allow_duplicate_keys=True)``
+    should produce exactly the same output as ``json.dumps(obj, fp).``
+    """
+
+    fp.write(str(dumps(obj=obj, skipkeys=skipkeys, ensure_ascii=ensure_ascii,
+                       check_circular=check_circular, allow_nan=allow_nan,
+                       cls=cls, indent=indent, separators=separators,
+                       default=default, sort_keys=sort_keys,
+                       quote_keys=quote_keys, trailing_commas=trailing_commas,
+                       allow_duplicate_keys=allow_duplicate_keys)))
+
+
+def dumps(obj, skipkeys=False, ensure_ascii=True, check_circular=True,
+          allow_nan=True, cls=None, indent=None, separators=None,
+          default=None, sort_keys=False,
+          quote_keys=False, trailing_commas=True, allow_duplicate_keys=True,
+          **kwargs):
+    """Serialize ``obj`` to a JSON5-formatted ``str``.
+
+    Supports the same arguments as ``json.dumps()``, except that:
+
+    - The ``cls`` keyword is not supported.
+    - The ``encoding`` keyword is ignored; Unicode strings are always returned.
+    - By default, object keys that are legal identifiers are not quoted;
+      if you pass quote_keys=True, they will be.
+    - By default, if lists and objects span multiple lines of output (i.e.,
+      when ``indent`` >=0), the last item will have a trailing comma
+      after it. If you pass ``trailing_commas=False, it will not.
+    - If you use a number, a boolean, or None as a key value in a dict,
+      it will be converted to the corresponding json string value, e.g.
+      "1", "true", or "null". By default, dump() will match the ``json``
+      module's behavior and produce ill-formed JSON if you mix keys of
+      different types that have the same converted value, e.g.:
+      {1: "foo", "1": "bar"} produces '{"1": "foo", "1": "bar"}', an
+      object with duplicated keys. If you pass ``allow_duplicate_keys=False``,
+      an exception will be raised instead.
+
 
+    Calling ``dumps(obj, quote_keys=True, trailing_commas=False,
+                    allow_duplicate_keys=True)``
+    should produce exactly the same output as ``json.dumps(obj).``
+    """
 
-def _dumpkey(k):
-    if _notletter.search(k):
-        return json.dumps(k)
+    assert kwargs.get('cls', None) is None, 'Custom encoders are not supported'
+
+    if separators is None:
+        if indent is None:
+            separators = (u', ', u': ')
+        else:
+            separators = (u',', u': ')
+
+    default = default or _raise_type_error
+
+    if check_circular:
+        seen = set()
     else:
-        return str(k)
+        seen = None
 
+    level = 1
+    is_key = False
 
-def dumps(obj, compact=False, as_json=False, **kwargs):
-    """Serialize ``obj`` to a JSON5-formatted ``str``."""
+    _, v = _dumps(obj, skipkeys, ensure_ascii, check_circular,
+                  allow_nan, indent, separators, default, sort_keys,
+                  quote_keys, trailing_commas, allow_duplicate_keys,
+                  seen, level, is_key)
+    return v
 
-    if as_json or not compact:
-        return json.dumps(obj, **kwargs)
+
+def _dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, indent,
+           separators, default, sort_keys,
+           quote_keys, trailing_commas, allow_duplicate_keys,
+           seen, level, is_key):
+    s = None
+    if obj is True:
+        s = u'true'
+    if obj is False:
+        s = u'false'
+    if obj is None:
+        s = u'null'
 
     t = type(obj)
-    if obj == True:
-        return u'true'
-    elif obj == False:
-        return u'false'
-    elif obj == None:
-        return u'null'
-    elif t == type('') or t == type(u''):
-        single = "'" in obj
-        double = '"' in obj
-        if single and double:
-            return json.dumps(obj)
-        elif single:
-            return '"' + obj + '"'
+    if t == type('') or t == type(u''):
+        if (is_key and _is_ident(obj) and not quote_keys
+            and not _is_reserved_word(obj)):
+            return True, obj
+        return True, _dump_str(obj, ensure_ascii)
+    if t is float:
+        s = _dump_float(obj,allow_nan)
+    if t is int:
+        s = str(obj)
+
+    if is_key:
+        if s is not None:
+            return True, '"%s"' % s
+        if skipkeys:
+            return False, None
+        raise TypeError('invalid key %s' % repr(obj))
+
+    if s is not None:
+        return True, s
+
+    if indent is not None:
+        end_str = ''
+        if trailing_commas:
+            end_str = ','
+        if type(indent) == int:
+            if indent > 0:
+                indent_str = '\n' + ' ' * indent * level
+                end_str += '\n' + ' ' * indent * (level - 1)
+            else:
+                indent_str = '\n'
+                end_str += '\n'
         else:
-            return "'" + obj + "'"
-    elif t is float or t is int:
-        return str(obj)
-    elif t is dict:
-        return u'{' + u','.join([
-            _dumpkey(k) + u':' + dumps(v) for k, v in obj.items()
-        ]) + '}'
-    elif t is list:
-        return u'[' + ','.join([dumps(el) for el in obj]) + u']'
-    else:  # pragma: no cover
-        return u''
-
-
-def dump(obj, fp, **kwargs):
-    """Serialize ``obj`` to a JSON5-formatted stream to ``fp`` (a ``.write()``-
-    supporting file-like object)."""
+            indent_str = '\n' + indent * level
+            end_str += '\n' + indent * (level - 1)
+    else:
+        indent_str = ''
+        end_str = ''
+
+    item_sep, kv_sep = separators
+    item_sep += indent_str
+    level += 1
+
+    if seen is not None:
+        i = id(obj)
+        if i in seen:
+            raise ValueError('Circular reference detected.')
+        else:
+            seen.add(i)
+
+    # In Python3, we'd check if this was an abc.Mapping or an abc.Sequence.
+    # For now, just check for the attrs we need to iterate over the object.
+    if hasattr(t, 'keys') and hasattr(t, '__getitem__'):
+        s = _dump_dict(obj, skipkeys, ensure_ascii,
+                       check_circular, allow_nan, indent,
+                       separators, default, sort_keys,
+                       quote_keys, trailing_commas,
+                       allow_duplicate_keys, seen, level,
+                       item_sep, kv_sep, indent_str, end_str)
+    elif hasattr(t, '__getitem__') and hasattr(t, '__iter__'):
+        s = _dump_array(obj, skipkeys, ensure_ascii,
+                        check_circular, allow_nan, indent,
+                        separators, default, sort_keys,
+                        quote_keys, trailing_commas,
+                        allow_duplicate_keys, seen, level,
+                        item_sep, indent_str, end_str)
+    else:
+        s = default(obj)
+
+    if seen is not None:
+        seen.remove(i)
+    return False, s
+
+
+def _dump_dict(obj, skipkeys, ensure_ascii, check_circular, allow_nan,
+               indent, separators, default, sort_keys,
+               quote_keys, trailing_commas, allow_duplicate_keys,
+               seen, level, item_sep, kv_sep, indent_str, end_str):
+    if not obj:
+        return u'{}'
+
+    if sort_keys:
+        keys = sorted(obj.keys())
+    else:
+        keys = obj.keys()
+
+    s = u'{' + indent_str
+
+    num_items_added = 0
+    new_keys = set()
+    for key in keys:
+        valid_key, key_str = _dumps(key, skipkeys, ensure_ascii, check_circular,
+                                    allow_nan, indent, separators, default,
+                                    sort_keys,
+                                    quote_keys, trailing_commas,
+                                    allow_duplicate_keys,
+                                    seen, level, is_key=True)
+        if valid_key:
+            if not allow_duplicate_keys:
+                if key_str in new_keys:
+                    raise ValueError('duplicate key %s' % repr(key))
+                else:
+                    new_keys.add(key_str)
+            if num_items_added:
+                s += item_sep
+            s += key_str + kv_sep + _dumps(obj[key], skipkeys, ensure_ascii,
+                                           check_circular, allow_nan, indent,
+                                           separators, default, sort_keys,
+                                           quote_keys, trailing_commas,
+                                           allow_duplicate_keys,
+                                           seen, level, is_key=False)[1]
+            num_items_added += 1
+        elif not skipkeys:
+            raise TypeError('invalid key %s' % repr(key))
+
+    s += end_str + u'}'
+    return s
+
+
+def _dump_array(obj, skipkeys, ensure_ascii, check_circular, allow_nan,
+                indent, separators, default, sort_keys,
+                quote_keys, trailing_commas, allow_duplicate_keys,
+                seen, level, item_sep, indent_str, end_str):
+    if not obj:
+        return u'[]'
+    return (u'[' + indent_str +
+            item_sep.join([_dumps(el, skipkeys, ensure_ascii, check_circular,
+                                  allow_nan, indent, separators, default,
+                                  sort_keys, quote_keys, trailing_commas,
+                                  allow_duplicate_keys,
+                                  seen, level, False)[1] for el in obj]) +
+            end_str + u']')
+
+
+def _dump_float(obj, allow_nan):
+    if allow_nan:
+        if math.isnan(obj):
+            return 'NaN'
+        if obj == float('inf'):
+            return 'Infinity'
+        if obj == float('-inf'):
+            return '-Infinity'
+    elif math.isnan(obj) or obj == float('inf') or obj == float('-inf'):
+        raise ValueError('Out of range float values '
+                         'are not JSON compliant')
+    return str(obj)
+
+
+def _dump_str(obj, ensure_ascii):
+    ret = ['"']
+    for ch in obj:
+        if ch == '\\':
+            ret.append('\\\\')
+        elif ch == '"':
+            ret.append('\\"')
+        elif ch == u'\u2028':
+            ret.append('\\u2028')
+        elif ch == u'\u2029':
+            ret.append('\\u2029')
+        elif ch == '\n':
+            ret.append('\\n')
+        elif ch == '\r':
+            ret.append('\\r')
+        elif ch == '\b':
+            ret.append('\\b')
+        elif ch == '\f':
+            ret.append('\\f')
+        elif ch == '\t':
+            ret.append('\\t')
+        elif ch == '\v':
+            ret.append('\\v')
+        elif ch == '\0':
+            ret.append('\\0')
+        elif not ensure_ascii:
+            ret.append(ch)
+        else:
+            o = ord(ch)
+            if o >= 32 and o < 128:
+                ret.append(ch)
+            elif o < 65536:
+                ret.append('\\u' + '%04x' % o)
+            else:
+                val = o - 0x10000
+                high = 0xd800 + (val >> 10)
+                low = 0xdc00 + (val & 0x3ff)
+                ret.append('\\u%04x\\u%04x' % (high, low))
+    return u''.join(ret) + '"'
+
+
+def _is_ident(k):
+    k = str(k)
+    if not k or not _is_id_start(k[0]) and k[0] not in (u'$', u'_'):
+        return False
+    for ch in k[1:]:
+        if not _is_id_continue(ch) and ch not in (u'$', u'_'):
+            return False
+    return True
+
+
+def _is_id_start(ch):
+    return unicodedata.category(ch) in (
+        'Lu', 'Ll', 'Li', 'Lt', 'Lm', 'Lo', 'Nl')
+
+
+def _is_id_continue(ch):
+    return unicodedata.category(ch) in (
+        'Lu', 'Ll', 'Li', 'Lt', 'Lm', 'Lo', 'Nl', 'Nd', 'Mn', 'Mc', 'Pc')
+
+
+_reserved_word_re = None
+
+def _is_reserved_word(k):
+    global _reserved_word_re
+
+    if _reserved_word_re is None:
+        # List taken from section 7.6.1 of ECMA-262.
+        _reserved_word_re = re.compile('(' + '|'.join([
+            'break',
+            'case',
+            'catch',
+            'class',
+            'const',
+            'continue',
+            'debugger',
+            'default',
+            'delete',
+            'do',
+            'else',
+            'enum',
+            'export',
+            'extends',
+            'false',
+            'finally',
+            'for',
+            'function',
+            'if',
+            'import',
+            'in',
+            'instanceof',
+            'new',
+            'null',
+            'return',
+            'super',
+            'switch',
+            'this',
+            'throw',
+            'true',
+            'try',
+            'typeof',
+            'var',
+            'void',
+            'while',
+            'with',
+        ]) + ')$')
+    return _reserved_word_re.match(k) is not None
+
 
-    s = dumps(obj, **kwargs)
-    fp.write(str(s))
+def _raise_type_error(obj):
+    raise TypeError('%s is not JSON5 serializable' % repr(obj))

third_party/pyjson5/src/json5/parser.py

-# pylint: disable=line-too-long
+# pylint: disable=line-too-long,unnecessary-lambda
 
 import sys
 
 
 if sys.version_info[0] < 3:
-    # pylint: disable=redefined-builtin
+    # pylint: disable=redefined-builtin,invalid-name
     chr = unichr
     range = xrange
     str = unicode
@@ -100,6 +100,8 @@ class Parser(object):
             rule()
             if self.failed:
                 self._rewind(p)
+                if p < self.errpos:
+                    self.errpos = p
                 break
             else:
                 vs.append(self.val)
@@ -127,12 +129,12 @@ class Parser(object):
         else:
             self._fail()
 
-    def _str(self, s, l):
-        p = self.pos
-        if (p + l <= self.end) and self.msg[p:p + l] == s:
-            self._succeed(s, self.pos + l)
-        else:
-            self._fail()
+    def _str(self, s):
+        for ch in s:
+            self._ch(ch)
+            if self.failed:
+                return
+        self.val = s
 
     def _range(self, i, j):
         p = self.pos
@@ -191,7 +193,7 @@ class Parser(object):
         self._ch('\f')
 
     def _ws__c6_(self):
-        self._ch('\u00a0')
+        self._ch(u'\xa0')
 
     def _ws__c7_(self):
         self._ch(u'\ufeff')
@@ -232,21 +234,21 @@ class Parser(object):
         self._choose([self._comment__c0_, self._comment__c1_])
 
     def _comment__c0_(self):
-        self._seq([lambda: self._str('//', 2),
+        self._seq([lambda: self._str('//'),
                    lambda: self._star(self._comment__c0__s1_p_)])
 
     def _comment__c0__s1_p_(self):
         self._seq([lambda: self._not(self._eol_), self._anything_])
 
     def _comment__c1_(self):
-        self._seq([lambda: self._str('/*', 2), self._comment__c1__s1_,
-                   lambda: self._str('*/', 2)])
+        self._seq([lambda: self._str('/*'), self._comment__c1__s1_,
+                   lambda: self._str('*/')])
 
     def _comment__c1__s1_(self):
         self._star(lambda: self._seq([self._comment__c1__s1_p__s0_, self._anything_]))
 
     def _comment__c1__s1_p__s0_(self):
-        self._not(lambda: self._str('*/', 2))
+        self._not(lambda: self._str('*/'))
 
     def _value_(self):
         self._choose([self._value__c0_, self._value__c1_, self._value__c2_,
@@ -254,14 +256,13 @@ class Parser(object):
                       self._value__c6_])
 
     def _value__c0_(self):
-        self._seq([lambda: self._str('null', 4), lambda: self._succeed('None')])
+        self._seq([lambda: self._str('null'), lambda: self._succeed('None')])
 
     def _value__c1_(self):
-        self._seq([lambda: self._str('true', 4), lambda: self._succeed('True')])
+        self._seq([lambda: self._str('true'), lambda: self._succeed('True')])
 
     def _value__c2_(self):
-        self._seq([lambda: self._str('false', 5),
-                   lambda: self._succeed('False')])
+        self._seq([lambda: self._str('false'), lambda: self._succeed('False')])
 
     def _value__c3_(self):
         self._push('value__c3')
@@ -393,7 +394,8 @@ class Parser(object):
                       self._esc_char__c4_, self._esc_char__c5_,
                       self._esc_char__c6_, self._esc_char__c7_,
                       self._esc_char__c8_, self._esc_char__c9_,
-                      self._esc_char__c10_])
+                      self._esc_char__c10_, self._esc_char__c11_,
+                      self._esc_char__c12_])
 
     def _esc_char__c0_(self):
         self._seq([lambda: self._ch('b'), lambda: self._succeed('\b')])
@@ -402,10 +404,20 @@ class Parser(object):
         self._seq([lambda: self._ch('f'), lambda: self._succeed('\f')])
 
     def _esc_char__c10_(self):
-        self._push('esc_char__c10')
+        self._seq([lambda: self._ch('0'), lambda: self._not(self._digit_),
+                   lambda: self._succeed('\x00')])
+
+    def _esc_char__c11_(self):
+        self._push('esc_char__c11')
+        self._seq([lambda: self._bind(self._hex_esc_, 'c'),
+                   lambda: self._succeed(self._get('c'))])
+        self._pop('esc_char__c11')
+
+    def _esc_char__c12_(self):
+        self._push('esc_char__c12')
         self._seq([lambda: self._bind(self._unicode_esc_, 'c'),
                    lambda: self._succeed(self._get('c'))])
-        self._pop('esc_char__c10')
+        self._pop('esc_char__c12')
 
     def _esc_char__c2_(self):
         self._seq([lambda: self._ch('n'), lambda: self._succeed('\n')])
@@ -430,10 +442,26 @@ class Parser(object):
 
     def _esc_char__c9_(self):
         self._push('esc_char__c9')
-        self._seq([lambda: self._bind(self._hex_esc_, 'c'),
+        self._seq([self._esc_char__c9__s0_,
+                   lambda: self._bind(self._anything_, 'c'),
                    lambda: self._succeed(self._get('c'))])
         self._pop('esc_char__c9')
 
+    def _esc_char__c9__s0_(self):
+        self._not(lambda: (self._esc_char__c9__s0_n_g_)())
+
+    def _esc_char__c9__s0_n_g_(self):
+        self._choose([self._esc_char__c9__s0_n_g__c0_,
+                      self._esc_char__c9__s0_n_g__c1_,
+                      lambda: self._seq([self._digit_]),
+                      lambda: self._seq([self._eol_])])
+
+    def _esc_char__c9__s0_n_g__c0_(self):
+        self._seq([lambda: self._ch('x')])
+
+    def _esc_char__c9__s0_n_g__c1_(self):
+        self._seq([lambda: self._ch('u')])
+
     def _hex_esc_(self):
         self._push('hex_esc')
         self._seq([lambda: self._ch('x'), lambda: self._bind(self._hex_, 'h1'),
@@ -501,22 +529,6 @@ class Parser(object):
                    lambda: self._succeed([self._get('k'), self._get('v')])])
         self._pop('member__c1')
 
-    def _member_list_(self):
-        self._push('member_list')
-        self._seq([lambda: self._bind(self._member_, 'm'),
-                   self._member_list__s1_, self._sp_, self._member_list__s3_,
-                   lambda: self._succeed([self._get('m')] + self._get('ms'))])
-        self._pop('member_list')
-
-    def _member_list__s1_(self):
-        self._bind(lambda: self._star(self._member_list__s1_l_p_), 'ms')
-
-    def _member_list__s1_l_p_(self):
-        self._seq([self._sp_, lambda: self._ch(','), self._sp_, self._member_])
-
-    def _member_list__s3_(self):
-        self._opt(lambda: self._ch(','))
-
     def _ident_(self):
         self._push('ident')
         self._seq([lambda: self._bind(self._id_start_, 'hd'), self._ident__s1_,
@@ -734,10 +746,10 @@ class Parser(object):
         self._opt(lambda: self._ch('+'))
 
     def _num_literal__c3_(self):
-        self._str('Infinity', 8)
+        self._str('Infinity')
 
     def _num_literal__c4_(self):
-        self._str('NaN', 3)
+        self._str('NaN')
 
     def _dec_literal_(self):
         self._choose([self._dec_literal__c0_, self._dec_literal__c1_,
@@ -815,7 +827,7 @@ class Parser(object):
         self._pop('hex_literal')
 
     def _hex_literal__s0_(self):
-        self._choose([lambda: self._str('0x', 2), lambda: self._str('0X', 2)])
+        self._choose([lambda: self._str('0x'), lambda: self._str('0X')])
 
     def _hex_literal__s1_(self):
         self._bind(lambda: self._plus(self._hex_), 'hs')
@@ -829,25 +841,6 @@ class Parser(object):
     def _hex__c1_(self):
         self._range('A', 'F')
 
-    def _hex_esc_(self):
-        self._push('hex_esc')
-        self._seq([lambda: self._ch('x'), lambda: self._bind(self._hex_, 'h1'),
-                   lambda: self._bind(self._hex_, 'h2'),
-                   lambda: self._succeed(self._xtou(self._get('h1') + self._get('h2')))])
-        self._pop('hex_esc')
-
-    def _hex_literal_(self):
-        self._push('hex_literal')
-        self._seq([self._hex_literal__s0_, self._hex_literal__s1_,
-                   lambda: self._succeed('0x' + self._join('', self._get('hs')))])
-        self._pop('hex_literal')
-
-    def _hex_literal__s0_(self):
-        self._choose([lambda: self._str('0x', 2), lambda: self._str('0X', 2)])
-
-    def _hex_literal__s1_(self):
-        self._bind(lambda: self._plus(self._hex_), 'hs')
-
     def _frac_(self):
         self._push('frac')
         self._seq([lambda: self._ch('.'), self._frac__s1_,

third_party/pyjson5/src/json5/tool.py

@@ -12,16 +12,20 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-"""Command-line tool to validate and pretty-print JSON5.
+"""A tool to parse and pretty-print JSON5.
 
-Usage::
+Usage:
 
     $ echo '{foo:"bar"}' | python -m json5.tool
-    { foo: "bar" }
-    $
+    {
+        foo: 'bar',
+    }
+    $ echo '{foo:"bar"}' | python -m json5.tool --as-json
+    {
+        "foo": "bar"
+    }
 """
 
-import os
 import sys
 
 from . import arg_parser
@@ -33,14 +37,35 @@ from .version import VERSION
 def main(argv=None, host=None):
     host = host or Host()
 
-    parser = arg_parser.ArgumentParser(host, prog='json5')
+    parser = arg_parser.ArgumentParser(host, prog='json5', desc=__doc__)
     parser.add_argument('-c', metavar='STR', dest='cmd',
-                        help='inline json5 string')
-    parser.add_argument('--json', dest='as_json', action='store_const',
+                        help='inline json5 string to read instead of '
+                             'reading from a file')
+    parser.add_argument('--as-json', dest='as_json', action='store_const',
                         const=True, default=False,
-                        help='output as json')
-    parser.add_argument('files', nargs='*', default=[],
-                        help=parser.SUPPRESS)
+                        help='output as JSON '
+                             '(same as --quote-keys --no-trailing-commas)')
+    parser.add_argument('--indent', dest='indent', default=4,
+                        help='amount to indent each line '
+                             '(default is 4 spaces)')
+    parser.add_argument('--quote-keys', action='store_true', default=False,
+                        help='quote all object keys')
+    parser.add_argument('--no-quote-keys', action='store_false',
+                        dest='quote_keys',
+                        help="don't quote object keys that are identifiers"
+                             " (this is the default)")
+    parser.add_argument('--trailing-commas', action='store_true',
+                        default=True,
+                        help='add commas after the last item in multi-line '
+                             'objects and arrays (this is the default)')
+    parser.add_argument('--no-trailing-commas', dest='trailing_commas',
+                        action='store_false',
+                        help='do not add commas after the last item in '
+                             'multi-line lists and objects')
+    parser.add_argument('file', metavar='FILE', nargs='?', default='-',
+                        help='optional file to read JSON5 document from; if '
+                             'not specified or "-", will read from stdin '
+                             'instead')
     args = parser.parse_args(argv)
 
     if parser.exit_status is not None:
@@ -52,10 +77,29 @@ def main(argv=None, host=None):
 
     if args.cmd:
         inp = args.cmd
+    elif args.file == '-':
+        inp = host.stdin.read()
     else:
-        inp = ''.join(host.fileinput(args.files))
+        inp = host.read_text_file(args.file)
 
-    host.print_(lib.dumps(lib.loads(inp), compact=True, as_json=args.as_json))
+    if args.indent == 'None':
+        args.indent = None
+    else:
+        try:
+            args.indent = int(args.indent)
+        except ValueError:
+            pass
+
+    if args.as_json:
+        args.quote_keys = True
+        args.trailing_commas = False
+
+    obj = lib.loads(inp)
+    s = lib.dumps(obj,
+                  indent=args.indent,
+                  quote_keys=args.quote_keys,
+                  trailing_commas=args.trailing_commas)
+    host.print_(s)
     return 0
 
 

third_party/pyjson5/src/json5/version.py

@@ -12,4 +12,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-VERSION = '0.6.1'
+VERSION = '0.9.5'

third_party/pyjson5/src/pylintrc

@@ -14,261 +14,44 @@
 
 [MASTER]
 
-# Specify a configuration file.
-#rcfile=
-
-# Python code to execute, usually for sys.path manipulation such as
-# pygtk.require().
-#init-hook=
-
-# Profiled execution.
-profile=no
-
-# Add files or directories to the blacklist. They should be base names, not
-# paths.
-ignore=CVS
-
 # Pickle collected data for later comparisons.
 persistent=yes
 
-# List of plugins (as comma separated values of python modules names) to load,
-# usually to register additional checkers.
-load-plugins=
-
-
 [MESSAGES CONTROL]
 
-# Enable the message, report, category or checker with the given id(s). You can
-# either give multiple identifier separated by comma (,) or put this option
-# multiple time.
-#enable=
-
-# Disable the message, report, category or checker with the given id(s). You
-# can either give multiple identifier separated by comma (,) or put this option
-# multiple time (only on the command line, not in the configuration file where
-# it should appear only once).
-# CHANGED:
-# C0111: Missing docstring
-# I0011: Locally disabling WNNNN
-# R0201: Method could be a function
-# R0801: Similar lines
-# W0141: Used builtin function 'map'
-# W0142: Used * or ** magic
-# W0511: TODO
-# W0703: Catch "Exception"
-disable=C0111,I0011,R0201,R0801,W0141,W0142,W0511,W0703
-
+disable=
+    broad-except,
+    global-statement,
+    locally-disabled,
+    missing-docstring,
+    no-self-use,
+    too-many-arguments,
+    too-few-public-methods,
+    too-many-branches,
+    too-many-instance-attributes,
+    too-many-locals,
+    too-many-public-methods,
+    too-many-return-statements,
+    unidiomatic-typecheck,
 
 [REPORTS]
 
-# Set the output format. Available formats are text, parseable, colorized, msvs
-# (visual studio) and html
-output-format=text
-
-# Include message's id in output
-include-ids=yes
-
-# Put messages in a separate file for each module / package specified on the
-# command line instead of printing them on stdout. Reports (if any) will be
-# written in a file name "pylint_global.[txt|html]".
-files-output=no
-
-# Tells whether to display a full report or only the messages
-# CHANGED:
 reports=no
 
-# Python expression which should return a note less than 10 (10 is the highest
-# note). You have access to the variables errors warning, statement which
-# respectively contain the number of errors / warnings messages and the total
-# number of statements analyzed. This is used by the global evaluation report
-# (RP0004).
-evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
-
-# Add a comment according to your evaluation note. This is used by the global
-# evaluation report (RP0004).
-comment=no
-
-
-[VARIABLES]
-
-# Tells whether we should check for unused import in __init__ files.
-init-import=no
-
-# A regular expression matching the beginning of the name of dummy variables
-# (i.e. not used).
-dummy-variables-rgx=_|dummy
-
-# List of additional names supposed to be defined in builtins. Remember that
-# you should avoid to define new builtins when possible.
-additional-builtins=
-
-
-[TYPECHECK]
-
-# Tells whether missing members accessed in mixin class should be ignored. A
-# mixin class is detected if its name ends with "mixin" (case insensitive).
-ignore-mixin-members=yes
-
-# List of classes names for which member attributes should not be checked
-# (useful for classes with attributes dynamically set).
-ignored-classes=
-
-# When zope mode is activated, add a predefined set of Zope acquired attributes
-# to generated-members.
-zope=no
-
-# List of members which are set dynamically and missed by pylint inference
-# system, and so shouldn't trigger E0201 when accessed. Python regular
-# expressions are accepted.
-generated-members=
-
-
-[MISCELLANEOUS]
-
-# List of note tags to take in consideration, separated by a comma.
-notes=FIXME,XXX,TODO
-
-
-[SIMILARITIES]
-
-# Minimum lines number of a similarity.
-min-similarity-lines=4
-
-# Ignore comments when computing similarities.
-ignore-comments=yes
-
-# Ignore docstrings when computing similarities.
-ignore-docstrings=yes
-
-
-[FORMAT]
-
-# Maximum number of characters on a single line.
-# max-line-length=200
-
-# Maximum number of lines in a module
-# max-module-lines=1000
-
-# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
-# tab).
-# CHANGED:
-indent-string='    '
-
-
 [BASIC]
 
-# Required attributes for module, separated by a comma
-required-attributes=
-
-# List of builtins function names that should not be used, separated by a comma
-bad-functions=map,filter,apply,input
-
-# Regular expression which should only match correct module names
-module-rgx=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
-
-# Regular expression which should only match correct module level names
-const-rgx=(([a-zA-Z_][a-zA-Z0-9_]*)|(__.*__))$
-
-# Regular expression which should only match correct class names
-class-rgx=[A-Z_][a-zA-Z0-9]+$
+# By default, pylint wants method names to be at most 31 chars long,
+# but we want to allow up to 49 to allow for longer test names.
+method-rgx=[a-zA-Z_][a-zA-Z0-9_]{0,48}$
 
-# Regular expression which should only match correct function names
-function-rgx=[a-z_][a-z0-9_]{0,40}$
+# By default, pylint only allows UPPER_CASE constants, but we want to
+# allow snake_case as well in some situations.
+const-rgx=[a-zA-Z_][a-zA-Z0-9_]{0,30}$
 
-# Regular expression which should only match correct method names
-method-rgx=[a-z_][a-z0-9_]{0,48}$
-
-# Regular expression which should only match correct instance attribute names
-attr-rgx=[a-z_][a-z0-9_]{0,30}$
-
-# Regular expression which should only match correct argument names
+# By default, pylint wants all parameter names to be at least two chars long,
+# but we want to allow single-char parameter names as well.
 argument-rgx=[a-z_][a-z0-9_]{0,30}$
 
-# Regular expression which should only match correct variable names
-variable-rgx=[a-zA-Z0-9_]{0,30}$
-
-# Regular expression which should only match correct list comprehension /
-# generator expression variable names
-inlinevar-rgx=[A-Za-z_][A-Za-z0-9_]*$
-
-# Good variable names which should always be accepted, separated by a comma
-good-names=i,j,k,ex,Run,_
-
-# Bad variable names which should always be refused, separated by a comma
-bad-names=foo,bar,baz,toto,tutu,tata
-
-# Regular expression which should only match functions or classes name which do
-# not require a docstring
-no-docstring-rgx=__.*__
-
-
-[DESIGN]
-
-# Maximum number of arguments for function / method
-max-args=8
-
-# Argument names that match this expression will be ignored. Default to name
-# with leading underscore
-ignored-argument-names=_.*
-
-# Maximum number of locals for function / method body
-max-locals=32
-
-# Maximum number of return / yield for function / method body
-max-returns=32
-
-# Maximum number of branch for function / method body
-max-branchs=32
-
-# Maximum number of statements in function / method body
-max-statements=65
-
-# Maximum number of parents for a class (see R0901).
-max-parents=7
-
-# Maximum number of attributes for a class (see R0902).
-max-attributes=16
-
-# Minimum number of public methods for a class (see R0903).
-min-public-methods=0
-
-# Maximum number of public methods for a class (see R0904).
-max-public-methods=100
-
-
-[CLASSES]
-
-# List of interface methods to ignore, separated by a comma. This is used for
-# instance to not check methods defines in Zope's Interface base class.
-ignore-iface-methods=isImplementedBy,deferred,extends,names,namesAndDescriptions,queryDescriptionFor,getBases,getDescriptionFor,getDoc,getName,getTaggedValue,getTaggedValueTags,isEqualOrExtendedBy,setTaggedValue,isImplementedByInstancesOf,adaptWith,is_implemented_by
-
-# List of method names used to declare (i.e. assign) instance attributes.
-defining-attr-methods=__init__,__new__,setUp
-
-# List of valid names for the first argument in a class method.
-valid-classmethod-first-arg=cls
-
-
-[IMPORTS]
-
-# Deprecated modules which should not be used, separated by a comma
-deprecated-modules=regsub,string,TERMIOS,Bastion,rexec
-
-# Create a graph of every (i.e. internal and external) dependencies in the
-# given file (report RP0402 must not be disabled)
-import-graph=
-
-# Create a graph of external dependencies in the given file (report RP0402 must
-# not be disabled)
-ext-import-graph=
-
-# Create a graph of internal dependencies in the given file (report RP0402 must
-# not be disabled)
-int-import-graph=
-
-
-[EXCEPTIONS]
-
-# Exceptions that will emit a warning when being caught. Defaults to
-# "Exception"
-overgeneral-exceptions=Exception
+# By default, pylint wants all variable names to be at least two chars long,
+# but we want to allow single-char variable names as well.
+variable-rgx=[a-z_][a-z0-9_]{0,30}$

third_party/pyjson5/src/run

@@ -3,16 +3,11 @@
 from __future__ import print_function
 
 import argparse
-import os
 import subprocess
 import sys
 
 
-is_python3 = bool(sys.version_info.major == 3)
-has_python34 = False
 verbose = False
-repo_dir = os.path.abspath(os.path.dirname(__file__))
-path_to_cov = os.path.join(repo_dir, 'tools', 'cov.py')
 
 
 def call(*args, **kwargs):
@@ -25,8 +20,6 @@ def call(*args, **kwargs):
 
 def main(argv):
     parser = argparse.ArgumentParser(prog='run')
-    parser.add_argument('--no3', action='store_true',
-                        help='Do not run the tests under Python 3.')
     parser.add_argument('-v', '--verbose', action='store_true')
     subps = parser.add_subparsers()
 
@@ -37,16 +30,9 @@ def main(argv):
     subp.set_defaults(func=run_clean)
 
     subp = subps.add_parser('coverage',
-                            help='Run the tests and report code coverage.')
+                            help='Run tests and report code coverage.')
     subp.set_defaults(func=run_coverage)
 
-    subp = subps.add_parser('develop',
-                             help='Install a symlinked package locally.')
-    subp.set_defaults(func=run_develop)
-    subp.add_argument('--system', action='store_true',
-                      help=('Install to the system site-package dir '
-                            'rather than the user\'s (requires root).'))
-
     subp = subps.add_parser('format',
                             help='Reformat the source code.')
     subp.set_defaults(func=run_format)
@@ -77,34 +63,31 @@ def main(argv):
     global verbose
     if args.verbose:
         verbose = True
-    global has_python34
-    if not args.no3:
-        try:
-            ver = subprocess.check_output(['python3', '--version'])
-            has_python34 = ver.split()[1] >= '3.4'
-        except:
-            pass
     args.func(args)
 
 
 def run_build(args):
+    del args
     call([sys.executable, 'setup.py', 'build', '--quiet'])
 
 
 def run_clean(args):
+    del args
     call(['git', 'clean', '-fxd'])
 
 
 def run_coverage(args):
-    call(['typ', '-c', 'json5'])
-
-
-def run_develop(args):
-    call([sys.executable, 'setup.py', 'develop'])
+    del args
+    call(['python', '-m', 'coverage', 'run', '-m', 'unittest',
+          'discover', '-p', '*_test.py'])
+    call(['python3', '-m', 'coverage', 'run', '--append', '-m', 'unittest',
+          'discover', '-p', '*_test.py'])
+    call([sys.executable, '-m', 'coverage', 'report', '--show-missing'])
 
 
 def run_format(args):
-    call('autopep8 --in-place *.py */*.py */*/*.py', shell=True)
+    del args
+    call('autopep8 --in-place *.py */*.py', shell=True)
 
 
 def run_help(args):
@@ -122,12 +105,14 @@ def run_install(args):
 
 
 def run_lint(args):
-    call('pylint --rcfile=pylintrc */*.py */*/*.py', shell=True)
-    call('pep8 *.py */*.py */*/*.py', shell=True)
+    del args
+    call('pylint --rcfile=pylintrc */*.py', shell=True)
 
 
 def run_tests(args):
-    call(['typ', 'json5'])
+    del args
+    call([sys.executable, '-m', 'unittest', 'discover',
+          '-p', '*_test.py'])
 
 
 if __name__ == '__main__':

third_party/pyjson5/src/setup.cfg

+[metadata]
+license_files = LICENSE.txt
+
 [bdist_wheel]
 universal=1

third_party/pyjson5/src/setup.py

@@ -23,15 +23,13 @@ if here not in sys.path:
 
 import json5
 
-with open(os.path.join(here, 'README.rst')) as fp:
-    readme = fp.read().strip()
+with open(os.path.join(here, 'README.md')) as fp:
+    long_description = fp.read()
 
-readme_lines = readme.splitlines()
 
 setup(
     name='json5',
-    packages=find_packages(),
-    package_data={'': ['../README.rst']},
+    packages=find_packages(exclude=['tests']),
     entry_points={
         'console_scripts': [
             'pyjson5=json5.tool:main',
@@ -39,11 +37,17 @@ setup(
     },
     install_requires=[
     ],
+    extras_require={
+        'dev': [
+            'hypothesis'
+        ]
+    },
     version=json5.VERSION,
     author='Dirk Pranke',
     author_email='dpranke@chromium.org',
-    description=readme_lines[3],
-    long_description=('\n' + '\n'.join(readme_lines)),
+    description=long_description.splitlines()[2],
+    long_description=long_description,
+    long_description_content_type='text/markdown',
     url='https://github.com/dpranke/pyjson5',
     license='Apache',
     classifiers=[

third_party/pyjson5/src/tests/host_fake.py

+# Copyright 2014 Dirk Pranke. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import io
+import sys
+
+if sys.version_info[0] < 3:
+    # pylint: disable=redefined-builtin, invalid-name
+    str = unicode
+
+
+class FakeHost(object):
+    # "too many instance attributes" pylint: disable=R0902
+    # "redefining built-in" pylint: disable=W0622
+    # "unused arg" pylint: disable=W0613
+
+    python_interpreter = 'python'
+
+    def __init__(self):
+        self.stdin = io.StringIO()
+        self.stdout = io.StringIO()
+        self.stderr = io.StringIO()
+        self.platform = 'linux2'
+        self.sep = '/'
+        self.dirs = set([])
+        self.files = {}
+        self.written_files = {}
+        self.last_tmpdir = None
+        self.current_tmpno = 0
+        self.cwd = '/tmp'
+
+    def abspath(self, *comps):
+        relpath = self.join(*comps)
+        if relpath.startswith('/'):
+            return relpath
+        return self.join(self.cwd, relpath)
+
+    def chdir(self, *comps):  # pragma: no cover
+        path = self.join(*comps)
+        if not path.startswith('/'):
+            path = self.join(self.cwd, path)
+        self.cwd = path
+
+    def dirname(self, path):
+        return '/'.join(path.split('/')[:-1])
+
+    def getcwd(self):
+        return self.cwd
+
+    def join(self, *comps):  # pragma: no cover
+        p = ''
+        for c in comps:
+            if c in ('', '.'):
+                continue
+            elif c.startswith('/'):
+                p = c
+            elif p:
+                p += '/' + c
+            else:
+                p = c
+
+        # Handle ./
+        p = p.replace('/./', '/')
+
+        # Handle ../
+        while '/..' in p:
+            comps = p.split('/')
+            idx = comps.index('..')
+            comps = comps[:idx-1] + comps[idx+1:]
+            p = '/'.join(comps)
+        return p
+
+    def maybe_mkdir(self, *comps):  # pragma: no cover
+        path = self.abspath(self.join(*comps))
+        if path not in self.dirs:
+            self.dirs.add(path)
+
+    def mkdtemp(self, suffix='', prefix='tmp', dir=None, **_kwargs):
+        if dir is None:
+            dir = self.sep + '__im_tmp'
+        curno = self.current_tmpno
+        self.current_tmpno += 1
+        self.last_tmpdir = self.join(dir, '%s_%u_%s' % (prefix, curno, suffix))
+        self.dirs.add(self.last_tmpdir)
+        return self.last_tmpdir
+
+    def print_(self, msg=u'', end=u'\n', stream=None):
+        stream = stream or self.stdout
+        stream.write(str(msg) + str(end))
+        stream.flush()
+
+    def read_text_file(self, *comps):
+        return self._read(comps)
+
+    def _read(self, comps):
+        return self.files[self.abspath(*comps)]
+
+    def remove(self, *comps):
+        path = self.abspath(*comps)
+        self.files[path] = None
+        self.written_files[path] = None
+
+    def rmtree(self, *comps):
+        path = self.abspath(*comps)
+        for f in self.files:
+            if f.startswith(path):
+                self.remove(f)
+        self.dirs.remove(path)
+
+    def write_text_file(self, path, contents):
+        self._write(path, contents)
+
+    def _write(self, path, contents):
+        full_path = self.abspath(path)
+        self.maybe_mkdir(self.dirname(full_path))
+        self.files[full_path] = contents
+        self.written_files[full_path] = contents

third_party/pyjson5/src/tests/host_test.py

+# Copyright 2019 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import io
+import unittest
+
+from json5.host import Host
+
+class HostTest(unittest.TestCase):
+    maxDiff = None
+
+    def test_directory_and_file_operations(self):
+        h = Host()
+        orig_cwd = h.getcwd()
+
+        try:
+            d = h.mkdtemp()
+            h.chdir(d)
+            h.write_text_file('foo', 'bar')
+            contents = h.read_text_file('foo')
+            self.assertEqual(contents, 'bar')
+            h.chdir('..')
+            h.rmtree(d)
+        finally:
+            h.chdir(orig_cwd)
+
+    def test_print(self):
+        s = io.StringIO()
+        h = Host()
+        h.print_('hello, world', stream=s)
+        self.assertEqual('hello, world\n', s.getvalue())
+
+
+if __name__ == '__main__':  # pragma: no cover
+    unittest.main()

third_party/pyjson5/src/tests/lib_test.py

+# Copyright 2015 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import io
+import json
+import math
+import os
+import sys
+import unittest
+
+from collections import OrderedDict
+from string import printable
+
+import json5
+import hypothesis.strategies as some
+
+from hypothesis import given
+
+some_json = some.recursive(
+    some.none() |
+    some.booleans() |
+    some.floats(allow_nan=False) |
+    some.text(printable),
+    lambda children: some.lists(children, min_size=1)
+    | some.dictionaries(some.text(printable), children, min_size=1),
+)
+
+class TestLoads(unittest.TestCase):
+    maxDiff = None
+
+    def check(self, s, obj):
+        self.assertEqual(json5.loads(s), obj)
+
+    def check_fail(self, s, err=None):
+        try:
+            json5.loads(s)
+            self.fail()  # pragma: no cover
+        except ValueError as e:
+            if err:
+                self.assertEqual(err, str(e))
+
+    def test_arrays(self):
+        self.check('[]', [])
+        self.check('[0]', [0])
+        self.check('[0,1]', [0, 1])
+        self.check('[ 0 , 1 ]', [0, 1])
+
+        try:
+            json5.loads('[ ,]')
+            self.fail()
+        except ValueError as e:
+            self.assertIn('Unexpected "," at column 3', str(e))
+
+    def test_bools(self):
+        self.check('true', True)
+        self.check('false', False)
+
+    def test_cls_is_not_supported(self):
+        self.assertRaises(AssertionError, json5.loads, '1', cls=lambda x: x)
+
+    def test_duplicate_keys_should_be_allowed(self):
+        self.assertEqual(json5.loads('{foo: 1, foo: 2}',
+                                     allow_duplicate_keys=True),
+                         {"foo": 2})
+
+    def test_duplicate_keys_should_be_allowed_by_default(self):
+        self.check('{foo: 1, foo: 2}', {"foo": 2})
+
+    def test_duplicate_keys_should_not_be_allowed(self):
+        self.assertRaises(ValueError, json5.loads, '{foo: 1, foo: 2}',
+                          allow_duplicate_keys=False)
+
+    def test_empty_strings_are_errors(self):
+        self.check_fail('', 'Empty strings are not legal JSON5')
+
+    def test_encoding(self):
+        if sys.version_info[0] < 3:
+            s = '"\xf6"'
+        else:
+            s = b'"\xf6"'
+        self.assertEqual(json5.loads(s, encoding='iso-8859-1'),
+                         u'\xf6')
+
+    def test_numbers(self):
+        # decimal literals
+        self.check('1', 1)
+        self.check('-1', -1)
+        self.check('+1', 1)
+
+        # hex literals
+        self.check('0xf', 15)
+        self.check('0xfe', 254)
+        self.check('0xfff', 4095)
+        self.check('0XABCD', 43981)
+        self.check('0x123456', 1193046)
+
+        # floats
+        self.check('1.5', 1.5)
+        self.check('1.5e3', 1500.0)
+        self.check('-0.5e-2', -0.005)
+
+        # names
+        self.check('Infinity', float('inf'))
+        self.check('-Infinity', float('-inf'))
+        self.assertTrue(math.isnan(json5.loads('NaN')))
+        self.assertTrue(math.isnan(json5.loads('-NaN')))
+
+        # syntax errors
+        self.check_fail('14d', '<string>:1 Unexpected "d" at column 3')
+
+    def test_identifiers(self):
+        self.check('{a: 1}', {'a': 1})
+        self.check('{$: 1}', {'$': 1})
+        self.check('{_: 1}', {'_': 1})
+        self.check('{a_b: 1}', {'a_b': 1})
+        self.check('{a$: 1}', {'a$': 1})
+
+        # This valid JavaScript but not valid JSON5; keys must be identifiers
+        # or strings.
+        self.check_fail('{1: 1}')
+
+    def test_identifiers_unicode(self):
+        self.check(u'{\xc3: 1}', {u'\xc3': 1})
+
+    def test_null(self):
+        self.check('null', None)
+
+    def test_object_hook(self):
+        hook = lambda d: [d]
+        self.assertEqual(json5.loads('{foo: 1}', object_hook=hook),
+                         [{"foo": 1}])
+
+    def test_object_pairs_hook(self):
+        hook = lambda pairs: pairs
+        self.assertEqual(json5.loads('{foo: 1, bar: 2}',
+                                     object_pairs_hook=hook),
+                         [('foo', 1), ('bar', 2)])
+
+    def test_objects(self):
+        self.check('{}', {})
+        self.check('{"foo": 0}', {"foo": 0})
+        self.check('{"foo":0,"bar":1}', {"foo": 0, "bar": 1})
+        self.check('{ "foo" : 0 , "bar" : 1 }', {"foo": 0, "bar": 1})
+
+    def test_parse_constant(self):
+        hook = lambda x: x
+        self.assertEqual(json5.loads('-Infinity', parse_constant=hook),
+                         '-Infinity')
+        self.assertEqual(json5.loads('NaN', parse_constant=hook),
+                         'NaN')
+
+    def test_parse_float(self):
+        hook = lambda x: x
+        self.assertEqual(json5.loads('1.0', parse_float=hook), '1.0')
+
+    def test_parse_int(self):
+        hook = lambda x, base=10: x
+        self.assertEqual(json5.loads('1', parse_int=hook), '1')
+
+    def test_sample_file(self):
+        path = os.path.join(os.path.dirname(__file__), '..', 'sample.json5')
+        with open(path) as fp:
+            obj = json5.load(fp)
+        self.assertEqual({
+            u'oh': [
+                u"we shouldn't forget",
+                u"arrays can have",
+                u"trailing commas too",
+            ],
+            u"this": u"is a multi-line string",
+            u"delta": 10,
+            u"hex": 3735928559,
+            u"finally": "a trailing comma",
+            u"here": "is another",
+            u"to": float("inf"),
+            u"while": True,
+            u"half": 0.5,
+            u"foo": u"bar"
+            }, obj)
+
+    def test_strings(self):
+        self.check('"foo"', 'foo')
+        self.check("'foo'", 'foo')
+
+        # escape chars
+        self.check("'\\b\\t\\f\\n\\r\\v\\\\'", '\b\t\f\n\r\v\\')
+        self.check("'\\''", "'")
+        self.check('"\\""', '"')
+
+        # hex literals
+        self.check('"\\x66oo"', 'foo')
+
+        # unicode literals
+        self.check('"\\u0066oo"', 'foo')
+
+        # string literals w/ continuation markers at the end of the line.
+        # These should not have spaces is the result.
+        self.check('"foo\\\nbar"', 'foobar')
+        self.check("'foo\\\nbar'", 'foobar')
+
+        # unterminated string literals.
+        self.check_fail('"\n')
+        self.check_fail("'\n")
+
+        # bad hex literals
+        self.check_fail("'\\x0'")
+        self.check_fail("'\\xj'")
+        self.check_fail("'\\x0j'")
+
+        # bad unicode literals
+        self.check_fail("'\\u0'")
+        self.check_fail("'\\u00'")
+        self.check_fail("'\\u000'")
+        self.check_fail("'\\u000j'")
+        self.check_fail("'\\u00j0'")
+        self.check_fail("'\\u0j00'")
+        self.check_fail("'\\uj000'")
+
+    def test_unrecognized_escape_char(self):
+        self.check(r'"\/"', '/')
+
+    def test_nul(self):
+        self.check(r'"\0"', '\x00')
+
+    def test_whitespace(self):
+        self.check('\n1', 1)
+        self.check('\r1', 1)
+        self.check('\r\n1', 1)
+        self.check('\t1', 1)
+        self.check('\v1', 1)
+        self.check(u'\uFEFF 1', 1)
+        self.check(u'\u00A0 1', 1)
+        self.check(u'\u2028 1', 1)
+        self.check(u'\u2029 1', 1)
+
+
+class TestDump(unittest.TestCase):
+    def test_basic(self):
+        sio = io.StringIO()
+        json5.dump(True, sio)
+        self.assertEqual('true', sio.getvalue())
+
+
+class TestDumps(unittest.TestCase):
+    maxDiff = None
+
+    def check(self, obj, s):
+        self.assertEqual(json5.dumps(obj), s)
+
+    def test_allow_duplicate_keys(self):
+        self.assertIn(json5.dumps({1: "foo", "1": "bar"}),
+                      {'{"1": "foo", "1": "bar"}',
+                       '{"1": "bar", "1": "foo"}'})
+
+        self.assertRaises(ValueError, json5.dumps,
+                          {1: "foo", "1": "bar"},
+                           allow_duplicate_keys=False)
+
+    def test_arrays(self):
+        self.check([], '[]')
+        self.check([1, 2, 3], '[1, 2, 3]')
+        self.check([{'foo': 'bar'}, {'baz': 'quux'}],
+                    '[{foo: "bar"}, {baz: "quux"}]')
+
+    def test_bools(self):
+        self.check(True, 'true')
+        self.check(False, 'false')
+
+    def test_check_circular(self):
+        # This tests a trivial cycle.
+        l = [1, 2, 3]
+        l[2] = l
+        self.assertRaises(ValueError, json5.dumps, l)
+
+        # This checks that json5 doesn't raise an error. However,
+        # the underlying Python implementation likely will.
+        try:
+            json5.dumps(l, check_circular=False)
+            self.fail()  # pragma: no cover
+        except Exception as e:
+            self.assertNotIn(str(e), 'Circular reference detected')
+
+        # This checks that repeated but non-circular references
+        # are okay.
+        x = [1, 2]
+        y = {"foo": x, "bar": x}
+        self.check(y,
+                   '{foo: [1, 2], bar: [1, 2]}')
+
+        # This tests a more complicated cycle.
+        x = {}
+        y = {}
+        z = {}
+        z['x'] = x
+        z['y'] = y
+        z['x']['y'] = y
+        z['y']['x'] = x
+        self.assertRaises(ValueError, json5.dumps, z)
+
+    def test_default(self):
+
+        def _custom_serializer(obj):
+            del obj
+            return 'something'
+
+        self.assertRaises(TypeError, json5.dumps, set())
+        self.assertEqual(json5.dumps(set(), default=_custom_serializer),
+                         'something')
+
+    def test_ensure_ascii(self):
+        self.check(u'\u00fc', '"\\u00fc"')
+        self.assertEqual(json5.dumps(u'\u00fc', ensure_ascii=False),
+                         u'"\u00fc"')
+
+    def test_indent(self):
+        self.assertEqual(json5.dumps([1, 2, 3], indent=None),
+                         u'[1, 2, 3]')
+        self.assertEqual(json5.dumps([1, 2, 3], indent=-1),
+                         u'[\n1,\n2,\n3,\n]')
+        self.assertEqual(json5.dumps([1, 2, 3], indent=0),
+                         u'[\n1,\n2,\n3,\n]')
+        self.assertEqual(json5.dumps([], indent=2),
+                         u'[]')
+        self.assertEqual(json5.dumps([1, 2, 3], indent=2),
+                         u'[\n  1,\n  2,\n  3,\n]')
+        self.assertEqual(json5.dumps([1, 2, 3], indent=' '),
+                         u'[\n 1,\n 2,\n 3,\n]')
+        self.assertEqual(json5.dumps([1, 2, 3], indent='++'),
+                         u'[\n++1,\n++2,\n++3,\n]')
+        self.assertEqual(json5.dumps([[1, 2, 3]], indent=2),
+                         u'[\n  [\n    1,\n    2,\n    3,\n  ],\n]')
+
+        self.assertEqual(json5.dumps({}, indent=2),
+                         u'{}')
+        self.assertEqual(json5.dumps({'foo': 'bar', 'baz': 'quux'}, indent=2),
+                         u'{\n  foo: "bar",\n  baz: "quux",\n}')
+
+    def test_numbers(self):
+        self.check(15, '15')
+        self.check(1.0, '1.0')
+        self.check(float('inf'), 'Infinity')
+        self.check(float('-inf'), '-Infinity')
+        self.check(float('nan'), 'NaN')
+
+        self.assertRaises(ValueError, json5.dumps,
+                          float('inf'), allow_nan=False)
+
+    def test_null(self):
+        self.check(None, 'null')
+
+    def test_objects(self):
+        self.check({'foo': 1}, '{foo: 1}')
+        self.check({'foo bar': 1}, '{"foo bar": 1}')
+        self.check({'1': 1}, '{"1": 1}')
+
+    def test_reserved_words_in_object_keys_are_quoted(self):
+        self.check({'new': 1}, '{"new": 1}')
+
+    def test_identifiers_only_starting_with_reserved_words_are_not_quoted(self):
+        self.check({'newbie': 1}, '{newbie: 1}')
+
+    def test_non_string_keys(self):
+        self.assertEqual(json5.dumps({False: 'a', 1: 'b', 2.0: 'c', None: 'd'}),
+                         '{"false": "a", "1": "b", "2.0": "c", "null": "d"}')
+
+    def test_quote_keys(self):
+        self.assertEqual(json5.dumps({"foo": 1}, quote_keys=True),
+                         '{"foo": 1}')
+
+    def test_strings(self):
+        self.check("'single'", '"\'single\'"')
+        self.check('"double"', '"\\"double\\""')
+        self.check("'single \\' and double \"'",
+                   '"\'single \\\\\' and double \\"\'"')
+
+    def test_string_escape_sequences(self):
+        self.check(u'\u2028\u2029\b\t\f\n\r\v\\\0',
+                   '"\\u2028\\u2029\\b\\t\\f\\n\\r\\v\\\\\\0"')
+
+    def test_skip_keys(self):
+        od = OrderedDict()
+        od[(1, 2)] = 2
+        self.assertRaises(TypeError, json5.dumps, od)
+        self.assertEqual(json5.dumps(od, skipkeys=True), '{}')
+
+        od['foo'] = 1
+        self.assertEqual(json5.dumps(od, skipkeys=True), '{foo: 1}')
+
+        # Also test that having an invalid key as the last element
+        # doesn't incorrectly add a trailing comma (see
+        # https://github.com/dpranke/pyjson5/issues/33).
+        od = OrderedDict()
+        od['foo'] = 1
+        od[(1, 2)] = 2
+        self.assertEqual(json5.dumps(od, skipkeys=True), '{foo: 1}')
+
+    def test_sort_keys(self):
+        od = OrderedDict()
+        od['foo'] = 1
+        od['bar'] = 2
+        self.assertEqual(json5.dumps(od, sort_keys=True),
+                         '{bar: 2, foo: 1}')
+
+    def test_trailing_commas(self):
+        # By default, multi-line dicts and lists should have trailing
+        # commas after their last items.
+        self.assertEqual(json5.dumps({"foo": 1}, indent=2),
+                         '{\n  foo: 1,\n}')
+        self.assertEqual(json5.dumps([1], indent=2),
+                         '[\n  1,\n]')
+
+        self.assertEqual(json5.dumps({"foo": 1}, indent=2,
+                                     trailing_commas=False),
+                         '{\n  foo: 1\n}')
+        self.assertEqual(json5.dumps([1], indent=2, trailing_commas=False),
+                         '[\n  1\n]')
+
+    def test_supplemental_unicode(self):
+        try:
+            s = chr(0x10000)
+            self.check(s, '"\\ud800\\udc00"')
+        except ValueError:
+            # Python2 doesn't support supplemental unicode planes, so
+            # we can't test this there.
+            pass
+
+    def test_empty_key(self):
+        self.assertEqual(json5.dumps({'': 'value'}), '{"": "value"}')
+
+    @given(some_json)
+    def test_object_roundtrip(self, input_object):
+        dumped_string_json = json.dumps(input_object)
+        dumped_string_json5 = json5.dumps(input_object)
+
+        parsed_object_json = json5.loads(dumped_string_json)
+        parsed_object_json5 = json5.loads(dumped_string_json5)
+
+        assert parsed_object_json == input_object
+        assert parsed_object_json5 == input_object
+
+
+if __name__ == '__main__':  # pragma: no cover
+    unittest.main()

third_party/pyjson5/src/tests/tool_test.py

+# Copyright 2017 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import sys
+import unittest
+
+import json5
+import json5.tool
+
+from .host_fake import FakeHost
+
+
+if sys.version_info[0] < 3:
+    # pylint: disable=redefined-builtin, invalid-name
+    str = unicode
+
+
+class CheckMixin(object):
+    def _write_files(self, host, files):
+        for path, contents in list(files.items()):
+            host.write_text_file(path, contents)
+
+    def check_cmd(self, args, stdin=None, files=None,
+                  returncode=None, out=None, err=None):
+        host = self._host()
+        orig_wd, tmpdir = None, None
+        try:
+            orig_wd = host.getcwd()
+            tmpdir = host.mkdtemp()
+            host.chdir(tmpdir)
+            if files:
+                self._write_files(host, files)
+            rv = self._call(host, args, stdin, returncode, out, err)
+            actual_ret, actual_out, actual_err = rv
+        finally:
+            if tmpdir:
+                host.rmtree(tmpdir)
+            if orig_wd:
+                host.chdir(orig_wd)
+
+        return actual_ret, actual_out, actual_err
+
+
+class UnitTestMixin(object):
+    def _host(self):
+        return FakeHost()
+
+    def _call(self, host, args, stdin=None,
+              returncode=None, out=None, err=None):
+        if stdin is not None:
+            host.stdin.write(str(stdin))
+            host.stdin.seek(0)
+        actual_ret = json5.tool.main(args, host)
+        actual_out = host.stdout.getvalue()
+        actual_err = host.stderr.getvalue()
+        if returncode is not None:
+            self.assertEqual(returncode, actual_ret)
+        if out is not None:
+            self.assertEqual(out, actual_out)
+        if err is not None:
+            self.assertEqual(err, actual_err)
+        return actual_ret, actual_out, actual_err
+
+
+class ToolTest(UnitTestMixin, CheckMixin, unittest.TestCase):
+    maxDiff = None
+
+    def test_help(self):
+        self.check_cmd(['--help'], returncode=0)
+
+    def test_inline_expression(self):
+        self.check_cmd(['-c', '{foo: 1}'], returncode=0,
+                       out=u'{\n    foo: 1,\n}\n')
+
+    def test_indent(self):
+        self.check_cmd(['--indent=None', '-c', '[1]'], returncode=0,
+                       out=u'[1]\n')
+        self.check_cmd(['--indent=2', '-c', '[1]'], returncode=0,
+                       out=u'[\n  1,\n]\n')
+        self.check_cmd(['--indent=  ', '-c', '[1]'], returncode=0,
+                       out=u'[\n  1,\n]\n')
+
+    def test_as_json(self):
+        self.check_cmd(['--as-json', '-c', '{foo: 1}'], returncode=0,
+                       out=u'{\n    "foo": 1\n}\n')
+
+    def test_quote_keys(self):
+        self.check_cmd(['--quote-keys', '-c', '{foo: 1}'], returncode=0,
+                       out=u'{\n    "foo": 1,\n}\n')
+
+    def test_no_quote_keys(self):
+        self.check_cmd(['--no-quote-keys', '-c', '{foo: 1}'], returncode=0,
+                       out=u'{\n    foo: 1,\n}\n')
+
+    def test_keys_are_quoted_by_default(self):
+        self.check_cmd(['-c', '{foo: 1}'], returncode=0,
+                       out=u'{\n    foo: 1,\n}\n')
+
+    def test_read_command(self):
+        self.check_cmd(['-c', '"foo"'], returncode=0, out=u'"foo"\n')
+
+    def test_read_from_stdin(self):
+        self.check_cmd([], stdin='"foo"\n', returncode=0, out=u'"foo"\n')
+
+    def test_read_from_a_file(self):
+        files = {
+            'foo.json5': '"foo"\n',
+        }
+        self.check_cmd(['foo.json5'], files=files, returncode=0, out=u'"foo"\n')
+
+    def test_trailing_commas(self):
+        self.check_cmd(['--trailing-commas', '-c', '{foo: 1}'], returncode=0,
+                       out=u'{\n    foo: 1,\n}\n')
+
+    def test_no_trailing_commas(self):
+        self.check_cmd(['--no-trailing-commas', '-c', '{foo: 1}'], returncode=0,
+                       out=u'{\n    foo: 1\n}\n')
+
+    def test_trailing_commas_are_there_by_default(self):
+        self.check_cmd(['-c', '{foo: 1}'], returncode=0,
+                       out=u'{\n    foo: 1,\n}\n')
+
+    def test_unknown_switch(self):
+        self.check_cmd(['--unknown-switch'], returncode=2,
+                       err=u'json5: error: unrecognized arguments: '
+                       '--unknown-switch\n\n')
+
+    def test_version(self):
+        self.check_cmd(['--version'], returncode=0,
+                       out=str(json5.VERSION) + '\n')
+
+
+if __name__ == '__main__':  # pragma: no cover
+    unittest.main()