Roll //thirdparty/pyjson5 to v0.9.5.

The current version of pyjson5 was v0.6.1, more than two years old :(.

There's no real significant new functionality in this new version and
the performance is still slow, but there are a number of bug fixes.

R=aboxhall@chromium.org

Change-Id: Ic293163382a1b0fa1111aca5b04a455eb9b72a45
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2216513
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#776381}

third_party/blink/renderer/bindings/scripts/build_web_idl_database.pydeps

 # Generated by running:
 #   build/print_python_deps.py --root third_party/blink/renderer/bindings/scripts --output third_party/blink/renderer/bindings/scripts/build_web_idl_database.pydeps third_party/blink/renderer/bindings/scripts/build_web_idl_database.py
 ../../../../pyjson5/src/json5/__init__.py
-../../../../pyjson5/src/json5/arg_parser.py
-../../../../pyjson5/src/json5/host.py
 ../../../../pyjson5/src/json5/lib.py
 ../../../../pyjson5/src/json5/parser.py
-../../../../pyjson5/src/json5/tool.py
 ../../../../pyjson5/src/json5/version.py
 ../../build/scripts/blinkbuild/__init__.py
 ../../build/scripts/blinkbuild/name_style_converter.py

third_party/blink/renderer/bindings/scripts/collect_idl_files.pydeps

@@ -8,11 +8,8 @@
 ../../../../ply/lex.py
 ../../../../ply/yacc.py
 ../../../../pyjson5/src/json5/__init__.py
-../../../../pyjson5/src/json5/arg_parser.py
-../../../../pyjson5/src/json5/host.py
 ../../../../pyjson5/src/json5/lib.py
 ../../../../pyjson5/src/json5/parser.py
-../../../../pyjson5/src/json5/tool.py
 ../../../../pyjson5/src/json5/version.py
 ../../build/scripts/blinkbuild/__init__.py
 ../../build/scripts/blinkbuild/name_style_converter.py

third_party/blink/renderer/bindings/scripts/generate_bindings.pydeps

@@ -20,11 +20,8 @@
 ../../../../markupsafe/_compat.py
 ../../../../markupsafe/_native.py
 ../../../../pyjson5/src/json5/__init__.py
-../../../../pyjson5/src/json5/arg_parser.py
-../../../../pyjson5/src/json5/host.py
 ../../../../pyjson5/src/json5/lib.py
 ../../../../pyjson5/src/json5/parser.py
-../../../../pyjson5/src/json5/tool.py
 ../../../../pyjson5/src/json5/version.py
 ../../build/scripts/blinkbuild/__init__.py
 ../../build/scripts/blinkbuild/name_style_converter.py

third_party/blink/renderer/bindings/scripts/generate_high_entropy_list.pydeps

 # Generated by running:
 #   build/print_python_deps.py --root third_party/blink/renderer/bindings/scripts --output third_party/blink/renderer/bindings/scripts/generate_high_entropy_list.pydeps third_party/blink/renderer/bindings/scripts/generate_high_entropy_list.py
 ../../../../pyjson5/src/json5/__init__.py
-../../../../pyjson5/src/json5/arg_parser.py
-../../../../pyjson5/src/json5/host.py
 ../../../../pyjson5/src/json5/lib.py
 ../../../../pyjson5/src/json5/parser.py
-../../../../pyjson5/src/json5/tool.py
 ../../../../pyjson5/src/json5/version.py
 ../../build/scripts/blinkbuild/__init__.py
 ../../build/scripts/blinkbuild/name_style_converter.py

third_party/pyjson5/README.chromium

 Name: pyjson5
 Short Name: pyjson5
 URL: https://github.com/dpranke/pyjson5
-Version: 0.6.1
-Date: May 22, 2018
-Revision: c88ac1f
+Version: 0.9.5
+Date: May 26 2020
+Revision: 9335da8
 License: Apache 2.0
 License File: src/LICENSE
 Security Critical: No

third_party/pyjson5/src/MANIFEST.in

+include *.md
+include LICENSE

third_party/pyjson5/src/README.md

+# pyjson5
+
+A Python implementation of the JSON5 data format.
+
+[JSON5](https://json5.org) extends the
+[JSON](http://www.json.org) data interchange format to make it
+slightly more usable as a configuration language:
+
+* JavaScript-style comments (both single and multi-line) are legal.
+
+* Object keys may be unquoted if they are legal ECMAScript identifiers
+
+* Objects and arrays may end with trailing commas.
+
+* Strings can be single-quoted, and multi-line string literals are allowed.
+
+There are a few other more minor extensions to JSON; see the above page for
+the full details.
+
+This project implements a reader and writer implementation for Python;
+where possible, it mirrors the
+[standard Python JSON API](https://docs.python.org/library/json.html)
+package for ease of use.
+
+There is one notable difference from the JSON api: the `load()` and
+`loads()` methods support optionally checking for (and rejecting) duplicate
+object keys; pass `allow_duplicate_keys=False` to do so (duplicates are
+allowed by default).
+
+This is an early release. It has been reasonably well-tested, but it is
+**SLOW**. It can be 1000-6000x slower than the C-optimized JSON module,
+and is 200x slower (or more) than the pure Python JSON module.
+
+## Known issues
+
+* Did I mention that it is **SLOW**?
+
+* The implementation follows Python3's `json` implementation where
+  possible. This means that the `encoding` method to `dump()` is
+  ignored, and unicode strings are always returned.
+
+* The `cls` keyword argument that `json.load()`/`json.loads()` accepts
+  to specify a custom subclass of ``JSONDecoder`` is not and will not be
+  supported, because this implementation uses a completely different
+  approach to parsing strings and doesn't have anything like the
+  `JSONDecoder` class.
+
+* The `cls` keyword argument that `json.dump()`/`json.dumps()` accepts
+  is also not supported, for consistency with `json5.load()`. The `default`
+  keyword *is* supported, though, and might be able to serve as a
+  workaround.
+
+## Running the tests
+To run the tests, setup a venv and install the required dependencies with
+`pip install -e '.[dev]'`, then run the tests with `python setup.py test`.
+
+
+## Version History / Release Notes
+
+* v0.9.5 (2020-05-26)
+    * Miscellaneous non-source cleanups in the repo, including setting
+      up GitHub Actions for a CI system. No changes to the library from
+      v0.9.4, other than updating the version.
+
+* v0.9.4 (2020-03-26)
+    * [GitHub pull #38](https://github.com/dpranke/pyjson5/pull/38)
+      Fix from fredrik@fornwall.net for dumps() crashing when passed
+      an empty string as a key in an object.
+
+* v0.9.3 (2020-03-17)
+    * [GitHub pull #35](https://github.com/dpranke/pyjson5/pull/35)
+      Fix from pastelmind@ for dump() not passing the right args to dumps().
+    * Fix from p.skouzos@novafutur.com to remove the tests directory from
+      the setup call, making the package a bit smaller.
+
+* v0.9.2 (2020-03-02)
+    * [GitHub pull #34](https://github.com/dpranke/pyjson5/pull/34)
+      Fix from roosephu@ for a badly formatted nested list.
+
+* v0.9.1 (2020-02-09)
+    * [GitHub issue #33](https://github.com/dpranke/pyjson5/issues/33):
+       Fix stray trailing comma when dumping an object with an invalid key.
+
+* v0.9.0 (2020-01-30)
+    * [GitHub issue #29](https://github.com/dpranke/pyjson5/issues/29):
+       Fix an issue where objects keys that started with a reserved
+       word were incorrectly quoted.
+    * [GitHub issue #30](https://github.com/dpranke/pyjson5/issues/30):
+       Fix an issue where dumps() incorrectly thought a data structure
+       was cyclic in some cases.
+    * [GitHub issue #32](https://github.com/dpranke/pyjson5/issues/32):
+       Allow for non-string keys in dicts passed to ``dump()``/``dumps()``.
+       Add an ``allow_duplicate_keys=False`` to prevent possible
+       ill-formed JSON that might result.
+
+* v0.8.5 (2019-07-04)
+    * [GitHub issue #25](https://github.com/dpranke/pyjson5/issues/25):
+      Add LICENSE and README.md to the dist.
+    * [GitHub issue #26](https://github.com/dpranke/pyjson5/issues/26):
+      Fix printing of empty arrays and objects with indentation, fix
+      misreporting of the position on parse failures in some cases.
+
+* v0.8.4 (2019-06-11)
+    * Updated the version history, too.
+
+* v0.8.3 (2019-06-11)
+    * Tweaked the README, bumped the version, forgot to update the version
+      history :).
+
+* v0.8.2 (2019-06-11)
+    * Actually bump the version properly, to 0.8.2.
+
+* v0.8.1 (2019-06-11)
+    * Fix bug in setup.py that messed up the description. Unfortunately,
+      I forgot to bump the version for this, so this also identifies as 0.8.0.
+
+* v0.8.0 (2019-06-11)
+    * Add `allow_duplicate_keys=True` as a default argument to
+      `json5.load()`/`json5.loads()`. If you set the key to `False`, duplicate
+      keys in a single dict will be rejected. The default is set to `True`
+      for compatibility with `json.load()`, earlier versions of json5, and
+      because it's simply not clear if people would want duplicate checking
+      enabled by default.
+
+* v0.7 (2019-03-31)
+    * Changes dump()/dumps() to not quote object keys by default if they are
+      legal identifiers. Passing `quote_keys=True` will turn that off
+      and always quote object keys.
+    * Changes dump()/dumps() to insert trailing commas after the last item
+      in an array or an object if the object is printed across multiple lines
+      (i.e., if `indent` is not None). Passing `trailing_commas=False` will
+      turn that off.
+    * The `json5.tool` command line tool now supports the `--indent`,
+      `--[no-]quote-keys`, and `--[no-]trailing-commas` flags to allow
+      for more control over the output, in addition to the existing
+      `--as-json` flag.
+    * The `json5.tool` command line tool no longer supports reading from
+      multiple files, you can now only read from a single file or
+      from standard input.
+    * The implementation no longer relies on the standard `json` module
+      for anything. The output should still match the json module (except
+      as noted above) and discrepancies should be reported as bugs.
+
+* v0.6.2 (2019-03-08)
+    * Fix [GitHub issue #23](https://github.com/dpranke/pyjson5/issues/23) and
+      pass through unrecognized escape sequences.
+
+* v0.6.1 (2018-05-22)
+    * Cleaned up a couple minor nits in the package.
+
+* v0.6.0 (2017-11-28)
+    * First implementation that attempted to implement 100% of the spec.
+
+* v0.5.0 (2017-09-04)
+    * First implementation that supported the full set of kwargs that
+      the `json` module supports.

third_party/pyjson5/src/benchmarks/chromium.linux.json

@@ -4133,15 +4133,15 @@
         }
       },
       {
-        "isolate_name": "devtools_lint_check",
-        "name": "devtools_lint_check",
+        "isolate_name": "devtools_closure_compile",
+        "name": "devtools_closure_compile",
         "swarming": {
           "can_use_on_swarming_builders": true
         }
       },
       {
-        "isolate_name": "devtools_type_check",
-        "name": "devtools_type_check",
+        "isolate_name": "devtools_eslint",
+        "name": "devtools_eslint",
         "swarming": {
           "can_use_on_swarming_builders": true
         }

third_party/pyjson5/src/benchmarks/mb_config.json

@@ -1966,7 +1966,7 @@
       "gn_args": "internal_gles2_conform_tests=true"
     }, 
     "java_coverage": {
-      "gn_args": "jacoco_coverage=true"
+      "gn_args": "emma_coverage=true emma_filter=\"org.chromium.*\""
     }, 
     "libfuzzer": {
       "gn_args": "use_libfuzzer=true"

third_party/pyjson5/src/benchmarks/run.py

@@ -26,7 +26,8 @@ REPO_DIR  = os.path.dirname(THIS_DIR)
 if not REPO_DIR in sys.path:
     sys.path.insert(0, REPO_DIR)
 
-import json5
+import json5  # pylint: disable=wrong-import-position
+
 
 ALL_BENCHMARKS = (
     'ios-simulator.json',
@@ -35,7 +36,9 @@ ALL_BENCHMARKS = (
     'chromium.perf.json',
 )
 
-DEFAULT_ITERATIONS = 1
+
+DEFAULT_ITERATIONS = 3
+
 
 def main():
     parser = argparse.ArgumentParser()
@@ -54,7 +57,9 @@ def main():
 
 
     # json.decoder.c_scanstring = py_scanstring
-    def py_maker(*_args, **_kwargs):
+    def py_maker(*args, **kwargs):
+        del args
+        del kwargs
         decoder = json.JSONDecoder()
         decoder.scan_once = json.scanner.py_make_scanner(decoder)
         decoder.parse_string = json.decoder.py_scanstring
@@ -65,7 +70,8 @@ def main():
 
     all_times = []
     for i, c in enumerate(file_contents):
-        times = []
+        json_time = 0.0
+        json5_time = 0.0
         for _ in range(args.num_iterations):
             start = time.time()
             json_obj = json.loads(c, cls=maker)
@@ -73,16 +79,30 @@ def main():
             json5_obj = json5.loads(c)
             end = time.time()
 
-            json_time = mid - start
-            json5_time = end - mid
-            times.append((json_time, json5_time))
-            assert(json5_obj == json_obj)
-        all_times.append(times)
-
-    for i, times in enumerate(all_times):
-        avg = sum((json5_time / json_time)
-                  for json_time, json5_time in times) / args.num_iterations
-        print("%-20s: %5.1f" % (args.benchmarks[i], avg))
+            json_time += mid - start
+            json5_time += end - mid
+            assert json5_obj == json_obj
+        all_times.append((json_time, json5_time))
+
+    for i, (json_time, json5_time) in enumerate(all_times):
+        fname = os.path.basename(args.benchmarks[i])
+        if json5_time and json_time:
+            if json5_time > json_time:
+                avg = json5_time / json_time
+                print("%-20s: JSON was %5.1fx faster (%.6fs to %.6fs)" % (
+                      fname, avg, json_time, json5_time))
+            else:
+                avg = json_time / json5_time
+                print("%-20s: JSON5 was %5.1fx faster (%.6fs to %.6fs)" % (
+                      fname, avg, json5_time, json_time))
+        elif json5_time:
+            print("%-20s: JSON5 took %.6f secs, JSON was too fast to measure" %
+                  (fname, json5_time))
+        elif json_time:
+            print("%-20s: JSON took %.6f secs, JSON5 was too fast to measure" %
+                  (fname, json_time))
+        else:
+            print("%-20s: both were too fast to measure" % (fname,))
 
     return 0
 

third_party/pyjson5/src/json5/__init__.py

@@ -14,7 +14,6 @@
 
 """A pure Python implementation of the JSON5 configuration language."""
 
-from . import tool
 from .lib import load, loads, dump, dumps
 from .version import VERSION
 
@@ -25,5 +24,4 @@ __all__ = [
     'dumps',
     'load',
     'loads',
-    'tool',
 ]

third_party/pyjson5/src/json5/arg_parser.py

@@ -22,12 +22,15 @@ class _Bailout(Exception):
 class ArgumentParser(argparse.ArgumentParser):
     SUPPRESS = argparse.SUPPRESS
 
-    def __init__(self, host, **kwargs):
+    def __init__(self, host, prog, desc, **kwargs):
+        kwargs['prog'] = prog
+        kwargs['description'] = desc
+        kwargs['formatter_class'] = argparse.RawDescriptionHelpFormatter
         super(ArgumentParser, self).__init__(**kwargs)
         self._host = host
         self.exit_status = None
         self.add_argument('-V', '--version', action='store_true',
-                          help='Print the version and exit.')
+                          help='print the version and exit')
 
     def parse_args(self, args=None, namespace=None):
         try:

third_party/pyjson5/src/json5/host.py

@@ -12,7 +12,6 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import fileinput
 import os
 import shutil
 import sys
@@ -20,7 +19,7 @@ import tempfile
 
 
 if sys.version_info[0] < 3:
-    # pylint: disable=redefined-builtin
+    # pylint: disable=redefined-builtin, invalid-name
     str = unicode
 
 
@@ -33,11 +32,6 @@ class Host(object):
     def chdir(self, *comps):
         return os.chdir(self.join(*comps))
 
-    def fileinput(self, files=None):
-        if not files:
-            return self.stdin.readlines()
-        return fileinput.input(files)
-
     def getcwd(self):
         return os.getcwd()
 
@@ -55,6 +49,10 @@ class Host(object):
     def rmtree(self, path):
         shutil.rmtree(path, ignore_errors=True)
 
+    def read_text_file(self, path):
+        with open(path, 'rb') as fp:
+            return fp.read().decode('utf8')
+
     def write_text_file(self, path, contents):
         with open(path, 'wb') as f:
             f.write(contents.encode('utf8'))

third_party/pyjson5/src/json5/json5.g

@@ -52,6 +52,8 @@ esc_char       = 'b'                                 -> '\u0008'
                | squote                              -> '\u0027'
                | dquote                              -> '\u0022'
                | bslash                              -> '\u005C'
+               | ~('x'|'u'|digit|eol) anything:c     -> c
+               | '0' ~digit                          -> '\u0000'
                | hex_esc:c                           -> c
                | unicode_esc:c                       -> c
 

third_party/pyjson5/src/json5/parser.py

-# pylint: disable=line-too-long
+# pylint: disable=line-too-long,unnecessary-lambda
 
 import sys
 
 
 if sys.version_info[0] < 3:
-    # pylint: disable=redefined-builtin
+    # pylint: disable=redefined-builtin,invalid-name
     chr = unichr
     range = xrange
     str = unicode
@@ -100,6 +100,8 @@ class Parser(object):
             rule()
             if self.failed:
                 self._rewind(p)
+                if p < self.errpos:
+                    self.errpos = p
                 break
             else:
                 vs.append(self.val)
@@ -127,12 +129,12 @@ class Parser(object):
         else:
             self._fail()
 
-    def _str(self, s, l):
-        p = self.pos
-        if (p + l <= self.end) and self.msg[p:p + l] == s:
-            self._succeed(s, self.pos + l)
-        else:
-            self._fail()
+    def _str(self, s):
+        for ch in s:
+            self._ch(ch)
+            if self.failed:
+                return
+        self.val = s
 
     def _range(self, i, j):
         p = self.pos
@@ -191,7 +193,7 @@ class Parser(object):
         self._ch('\f')
 
     def _ws__c6_(self):
-        self._ch('\u00a0')
+        self._ch(u'\xa0')
 
     def _ws__c7_(self):
         self._ch(u'\ufeff')
@@ -232,21 +234,21 @@ class Parser(object):
         self._choose([self._comment__c0_, self._comment__c1_])
 
     def _comment__c0_(self):
-        self._seq([lambda: self._str('//', 2),
+        self._seq([lambda: self._str('//'),
                    lambda: self._star(self._comment__c0__s1_p_)])
 
     def _comment__c0__s1_p_(self):
         self._seq([lambda: self._not(self._eol_), self._anything_])
 
     def _comment__c1_(self):
-        self._seq([lambda: self._str('/*', 2), self._comment__c1__s1_,
-                   lambda: self._str('*/', 2)])
+        self._seq([lambda: self._str('/*'), self._comment__c1__s1_,
+                   lambda: self._str('*/')])
 
     def _comment__c1__s1_(self):
         self._star(lambda: self._seq([self._comment__c1__s1_p__s0_, self._anything_]))
 
     def _comment__c1__s1_p__s0_(self):
-        self._not(lambda: self._str('*/', 2))
+        self._not(lambda: self._str('*/'))
 
     def _value_(self):
         self._choose([self._value__c0_, self._value__c1_, self._value__c2_,
@@ -254,14 +256,13 @@ class Parser(object):
                       self._value__c6_])
 
     def _value__c0_(self):
-        self._seq([lambda: self._str('null', 4), lambda: self._succeed('None')])
+        self._seq([lambda: self._str('null'), lambda: self._succeed('None')])
 
     def _value__c1_(self):
-        self._seq([lambda: self._str('true', 4), lambda: self._succeed('True')])
+        self._seq([lambda: self._str('true'), lambda: self._succeed('True')])
 
     def _value__c2_(self):
-        self._seq([lambda: self._str('false', 5),
-                   lambda: self._succeed('False')])
+        self._seq([lambda: self._str('false'), lambda: self._succeed('False')])
 
     def _value__c3_(self):
         self._push('value__c3')
@@ -393,7 +394,8 @@ class Parser(object):
                       self._esc_char__c4_, self._esc_char__c5_,
                       self._esc_char__c6_, self._esc_char__c7_,
                       self._esc_char__c8_, self._esc_char__c9_,
-                      self._esc_char__c10_])
+                      self._esc_char__c10_, self._esc_char__c11_,
+                      self._esc_char__c12_])
 
     def _esc_char__c0_(self):
         self._seq([lambda: self._ch('b'), lambda: self._succeed('\b')])
@@ -402,10 +404,20 @@ class Parser(object):
         self._seq([lambda: self._ch('f'), lambda: self._succeed('\f')])
 
     def _esc_char__c10_(self):
-        self._push('esc_char__c10')
+        self._seq([lambda: self._ch('0'), lambda: self._not(self._digit_),
+                   lambda: self._succeed('\x00')])
+
+    def _esc_char__c11_(self):
+        self._push('esc_char__c11')
+        self._seq([lambda: self._bind(self._hex_esc_, 'c'),
+                   lambda: self._succeed(self._get('c'))])
+        self._pop('esc_char__c11')
+
+    def _esc_char__c12_(self):
+        self._push('esc_char__c12')
         self._seq([lambda: self._bind(self._unicode_esc_, 'c'),
                    lambda: self._succeed(self._get('c'))])
-        self._pop('esc_char__c10')
+        self._pop('esc_char__c12')
 
     def _esc_char__c2_(self):
         self._seq([lambda: self._ch('n'), lambda: self._succeed('\n')])
@@ -430,10 +442,26 @@ class Parser(object):
 
     def _esc_char__c9_(self):
         self._push('esc_char__c9')
-        self._seq([lambda: self._bind(self._hex_esc_, 'c'),
+        self._seq([self._esc_char__c9__s0_,
+                   lambda: self._bind(self._anything_, 'c'),
                    lambda: self._succeed(self._get('c'))])
         self._pop('esc_char__c9')
 
+    def _esc_char__c9__s0_(self):
+        self._not(lambda: (self._esc_char__c9__s0_n_g_)())
+
+    def _esc_char__c9__s0_n_g_(self):
+        self._choose([self._esc_char__c9__s0_n_g__c0_,
+                      self._esc_char__c9__s0_n_g__c1_,
+                      lambda: self._seq([self._digit_]),
+                      lambda: self._seq([self._eol_])])
+
+    def _esc_char__c9__s0_n_g__c0_(self):
+        self._seq([lambda: self._ch('x')])
+
+    def _esc_char__c9__s0_n_g__c1_(self):
+        self._seq([lambda: self._ch('u')])
+
     def _hex_esc_(self):
         self._push('hex_esc')
         self._seq([lambda: self._ch('x'), lambda: self._bind(self._hex_, 'h1'),
@@ -501,22 +529,6 @@ class Parser(object):
                    lambda: self._succeed([self._get('k'), self._get('v')])])
         self._pop('member__c1')
 
-    def _member_list_(self):
-        self._push('member_list')
-        self._seq([lambda: self._bind(self._member_, 'm'),
-                   self._member_list__s1_, self._sp_, self._member_list__s3_,
-                   lambda: self._succeed([self._get('m')] + self._get('ms'))])
-        self._pop('member_list')
-
-    def _member_list__s1_(self):
-        self._bind(lambda: self._star(self._member_list__s1_l_p_), 'ms')
-
-    def _member_list__s1_l_p_(self):
-        self._seq([self._sp_, lambda: self._ch(','), self._sp_, self._member_])
-
-    def _member_list__s3_(self):
-        self._opt(lambda: self._ch(','))
-
     def _ident_(self):
         self._push('ident')
         self._seq([lambda: self._bind(self._id_start_, 'hd'), self._ident__s1_,
@@ -734,10 +746,10 @@ class Parser(object):
         self._opt(lambda: self._ch('+'))
 
     def _num_literal__c3_(self):
-        self._str('Infinity', 8)
+        self._str('Infinity')
 
     def _num_literal__c4_(self):
-        self._str('NaN', 3)
+        self._str('NaN')
 
     def _dec_literal_(self):
         self._choose([self._dec_literal__c0_, self._dec_literal__c1_,
@@ -815,7 +827,7 @@ class Parser(object):
         self._pop('hex_literal')
 
     def _hex_literal__s0_(self):
-        self._choose([lambda: self._str('0x', 2), lambda: self._str('0X', 2)])
+        self._choose([lambda: self._str('0x'), lambda: self._str('0X')])
 
     def _hex_literal__s1_(self):
         self._bind(lambda: self._plus(self._hex_), 'hs')
@@ -829,25 +841,6 @@ class Parser(object):
     def _hex__c1_(self):
         self._range('A', 'F')
 
-    def _hex_esc_(self):
-        self._push('hex_esc')
-        self._seq([lambda: self._ch('x'), lambda: self._bind(self._hex_, 'h1'),
-                   lambda: self._bind(self._hex_, 'h2'),
-                   lambda: self._succeed(self._xtou(self._get('h1') + self._get('h2')))])
-        self._pop('hex_esc')
-
-    def _hex_literal_(self):
-        self._push('hex_literal')
-        self._seq([self._hex_literal__s0_, self._hex_literal__s1_,
-                   lambda: self._succeed('0x' + self._join('', self._get('hs')))])
-        self._pop('hex_literal')
-
-    def _hex_literal__s0_(self):
-        self._choose([lambda: self._str('0x', 2), lambda: self._str('0X', 2)])
-
-    def _hex_literal__s1_(self):
-        self._bind(lambda: self._plus(self._hex_), 'hs')
-
     def _frac_(self):
         self._push('frac')
         self._seq([lambda: self._ch('.'), self._frac__s1_,

third_party/pyjson5/src/json5/tool.py

@@ -12,16 +12,20 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-"""Command-line tool to validate and pretty-print JSON5.
+"""A tool to parse and pretty-print JSON5.
 
-Usage::
+Usage:
 
     $ echo '{foo:"bar"}' | python -m json5.tool
-    { foo: "bar" }
-    $
+    {
+        foo: 'bar',
+    }
+    $ echo '{foo:"bar"}' | python -m json5.tool --as-json
+    {
+        "foo": "bar"
+    }
 """
 
-import os
 import sys
 
 from . import arg_parser
@@ -33,14 +37,35 @@ from .version import VERSION
 def main(argv=None, host=None):
     host = host or Host()
 
-    parser = arg_parser.ArgumentParser(host, prog='json5')
+    parser = arg_parser.ArgumentParser(host, prog='json5', desc=__doc__)
     parser.add_argument('-c', metavar='STR', dest='cmd',
-                        help='inline json5 string')
-    parser.add_argument('--json', dest='as_json', action='store_const',
+                        help='inline json5 string to read instead of '
+                             'reading from a file')
+    parser.add_argument('--as-json', dest='as_json', action='store_const',
                         const=True, default=False,
-                        help='output as json')
-    parser.add_argument('files', nargs='*', default=[],
-                        help=parser.SUPPRESS)
+                        help='output as JSON '
+                             '(same as --quote-keys --no-trailing-commas)')
+    parser.add_argument('--indent', dest='indent', default=4,
+                        help='amount to indent each line '
+                             '(default is 4 spaces)')
+    parser.add_argument('--quote-keys', action='store_true', default=False,
+                        help='quote all object keys')
+    parser.add_argument('--no-quote-keys', action='store_false',
+                        dest='quote_keys',
+                        help="don't quote object keys that are identifiers"
+                             " (this is the default)")
+    parser.add_argument('--trailing-commas', action='store_true',
+                        default=True,
+                        help='add commas after the last item in multi-line '
+                             'objects and arrays (this is the default)')
+    parser.add_argument('--no-trailing-commas', dest='trailing_commas',
+                        action='store_false',
+                        help='do not add commas after the last item in '
+                             'multi-line lists and objects')
+    parser.add_argument('file', metavar='FILE', nargs='?', default='-',
+                        help='optional file to read JSON5 document from; if '
+                             'not specified or "-", will read from stdin '
+                             'instead')
     args = parser.parse_args(argv)
 
     if parser.exit_status is not None:
@@ -52,10 +77,29 @@ def main(argv=None, host=None):
 
     if args.cmd:
         inp = args.cmd
+    elif args.file == '-':
+        inp = host.stdin.read()
     else:
-        inp = ''.join(host.fileinput(args.files))
+        inp = host.read_text_file(args.file)
 
-    host.print_(lib.dumps(lib.loads(inp), compact=True, as_json=args.as_json))
+    if args.indent == 'None':
+        args.indent = None
+    else:
+        try:
+            args.indent = int(args.indent)
+        except ValueError:
+            pass
+
+    if args.as_json:
+        args.quote_keys = True
+        args.trailing_commas = False
+
+    obj = lib.loads(inp)
+    s = lib.dumps(obj,
+                  indent=args.indent,
+                  quote_keys=args.quote_keys,
+                  trailing_commas=args.trailing_commas)
+    host.print_(s)
     return 0
 
 

third_party/pyjson5/src/json5/version.py

@@ -12,4 +12,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-VERSION = '0.6.1'
+VERSION = '0.9.5'

third_party/pyjson5/src/pylintrc

@@ -14,261 +14,44 @@
 
 [MASTER]
 
-# Specify a configuration file.
-#rcfile=
-
-# Python code to execute, usually for sys.path manipulation such as
-# pygtk.require().
-#init-hook=
-
-# Profiled execution.
-profile=no
-
-# Add files or directories to the blacklist. They should be base names, not
-# paths.
-ignore=CVS
-
 # Pickle collected data for later comparisons.
 persistent=yes
 
-# List of plugins (as comma separated values of python modules names) to load,
-# usually to register additional checkers.
-load-plugins=
-
-
 [MESSAGES CONTROL]
 
-# Enable the message, report, category or checker with the given id(s). You can
-# either give multiple identifier separated by comma (,) or put this option
-# multiple time.
-#enable=
-
-# Disable the message, report, category or checker with the given id(s). You
-# can either give multiple identifier separated by comma (,) or put this option
-# multiple time (only on the command line, not in the configuration file where
-# it should appear only once).
-# CHANGED:
-# C0111: Missing docstring
-# I0011: Locally disabling WNNNN
-# R0201: Method could be a function
-# R0801: Similar lines
-# W0141: Used builtin function 'map'
-# W0142: Used * or ** magic
-# W0511: TODO
-# W0703: Catch "Exception"
-disable=C0111,I0011,R0201,R0801,W0141,W0142,W0511,W0703
-
+disable=
+    broad-except,
+    global-statement,
+    locally-disabled,
+    missing-docstring,
+    no-self-use,
+    too-many-arguments,
+    too-few-public-methods,
+    too-many-branches,
+    too-many-instance-attributes,
+    too-many-locals,
+    too-many-public-methods,
+    too-many-return-statements,
+    unidiomatic-typecheck,
 
 [REPORTS]
 
-# Set the output format. Available formats are text, parseable, colorized, msvs
-# (visual studio) and html
-output-format=text
-
-# Include message's id in output
-include-ids=yes
-
-# Put messages in a separate file for each module / package specified on the
-# command line instead of printing them on stdout. Reports (if any) will be
-# written in a file name "pylint_global.[txt|html]".
-files-output=no
-
-# Tells whether to display a full report or only the messages
-# CHANGED:
 reports=no
 
-# Python expression which should return a note less than 10 (10 is the highest
-# note). You have access to the variables errors warning, statement which
-# respectively contain the number of errors / warnings messages and the total
-# number of statements analyzed. This is used by the global evaluation report
-# (RP0004).
-evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
-
-# Add a comment according to your evaluation note. This is used by the global
-# evaluation report (RP0004).
-comment=no
-
-
-[VARIABLES]
-
-# Tells whether we should check for unused import in __init__ files.
-init-import=no
-
-# A regular expression matching the beginning of the name of dummy variables
-# (i.e. not used).
-dummy-variables-rgx=_|dummy
-
-# List of additional names supposed to be defined in builtins. Remember that
-# you should avoid to define new builtins when possible.
-additional-builtins=
-
-
-[TYPECHECK]
-
-# Tells whether missing members accessed in mixin class should be ignored. A
-# mixin class is detected if its name ends with "mixin" (case insensitive).
-ignore-mixin-members=yes
-
-# List of classes names for which member attributes should not be checked
-# (useful for classes with attributes dynamically set).
-ignored-classes=
-
-# When zope mode is activated, add a predefined set of Zope acquired attributes
-# to generated-members.
-zope=no
-
-# List of members which are set dynamically and missed by pylint inference
-# system, and so shouldn't trigger E0201 when accessed. Python regular
-# expressions are accepted.
-generated-members=
-
-
-[MISCELLANEOUS]
-
-# List of note tags to take in consideration, separated by a comma.
-notes=FIXME,XXX,TODO
-
-
-[SIMILARITIES]
-
-# Minimum lines number of a similarity.
-min-similarity-lines=4
-
-# Ignore comments when computing similarities.
-ignore-comments=yes
-
-# Ignore docstrings when computing similarities.
-ignore-docstrings=yes
-
-
-[FORMAT]
-
-# Maximum number of characters on a single line.
-# max-line-length=200
-
-# Maximum number of lines in a module
-# max-module-lines=1000
-
-# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
-# tab).
-# CHANGED:
-indent-string='    '
-
-
 [BASIC]
 
-# Required attributes for module, separated by a comma
-required-attributes=
-
-# List of builtins function names that should not be used, separated by a comma
-bad-functions=map,filter,apply,input
-
-# Regular expression which should only match correct module names
-module-rgx=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
-
-# Regular expression which should only match correct module level names
-const-rgx=(([a-zA-Z_][a-zA-Z0-9_]*)|(__.*__))$
-
-# Regular expression which should only match correct class names
-class-rgx=[A-Z_][a-zA-Z0-9]+$
+# By default, pylint wants method names to be at most 31 chars long,
+# but we want to allow up to 49 to allow for longer test names.
+method-rgx=[a-zA-Z_][a-zA-Z0-9_]{0,48}$
 
-# Regular expression which should only match correct function names
-function-rgx=[a-z_][a-z0-9_]{0,40}$
+# By default, pylint only allows UPPER_CASE constants, but we want to
+# allow snake_case as well in some situations.
+const-rgx=[a-zA-Z_][a-zA-Z0-9_]{0,30}$
 
-# Regular expression which should only match correct method names
-method-rgx=[a-z_][a-z0-9_]{0,48}$
-
-# Regular expression which should only match correct instance attribute names
-attr-rgx=[a-z_][a-z0-9_]{0,30}$
-
-# Regular expression which should only match correct argument names
+# By default, pylint wants all parameter names to be at least two chars long,
+# but we want to allow single-char parameter names as well.
 argument-rgx=[a-z_][a-z0-9_]{0,30}$
 
-# Regular expression which should only match correct variable names
-variable-rgx=[a-zA-Z0-9_]{0,30}$
-
-# Regular expression which should only match correct list comprehension /
-# generator expression variable names
-inlinevar-rgx=[A-Za-z_][A-Za-z0-9_]*$
-
-# Good variable names which should always be accepted, separated by a comma
-good-names=i,j,k,ex,Run,_
-
-# Bad variable names which should always be refused, separated by a comma
-bad-names=foo,bar,baz,toto,tutu,tata
-
-# Regular expression which should only match functions or classes name which do
-# not require a docstring
-no-docstring-rgx=__.*__
-
-
-[DESIGN]
-
-# Maximum number of arguments for function / method
-max-args=8
-
-# Argument names that match this expression will be ignored. Default to name
-# with leading underscore
-ignored-argument-names=_.*
-
-# Maximum number of locals for function / method body
-max-locals=32
-
-# Maximum number of return / yield for function / method body
-max-returns=32
-
-# Maximum number of branch for function / method body
-max-branchs=32
-
-# Maximum number of statements in function / method body
-max-statements=65
-
-# Maximum number of parents for a class (see R0901).
-max-parents=7
-
-# Maximum number of attributes for a class (see R0902).
-max-attributes=16
-
-# Minimum number of public methods for a class (see R0903).
-min-public-methods=0
-
-# Maximum number of public methods for a class (see R0904).
-max-public-methods=100
-
-
-[CLASSES]
-
-# List of interface methods to ignore, separated by a comma. This is used for
-# instance to not check methods defines in Zope's Interface base class.
-ignore-iface-methods=isImplementedBy,deferred,extends,names,namesAndDescriptions,queryDescriptionFor,getBases,getDescriptionFor,getDoc,getName,getTaggedValue,getTaggedValueTags,isEqualOrExtendedBy,setTaggedValue,isImplementedByInstancesOf,adaptWith,is_implemented_by
-
-# List of method names used to declare (i.e. assign) instance attributes.
-defining-attr-methods=__init__,__new__,setUp
-
-# List of valid names for the first argument in a class method.
-valid-classmethod-first-arg=cls
-
-
-[IMPORTS]
-
-# Deprecated modules which should not be used, separated by a comma
-deprecated-modules=regsub,string,TERMIOS,Bastion,rexec
-
-# Create a graph of every (i.e. internal and external) dependencies in the
-# given file (report RP0402 must not be disabled)
-import-graph=
-
-# Create a graph of external dependencies in the given file (report RP0402 must
-# not be disabled)
-ext-import-graph=
-
-# Create a graph of internal dependencies in the given file (report RP0402 must
-# not be disabled)
-int-import-graph=
-
-
-[EXCEPTIONS]
-
-# Exceptions that will emit a warning when being caught. Defaults to
-# "Exception"
-overgeneral-exceptions=Exception
+# By default, pylint wants all variable names to be at least two chars long,
+# but we want to allow single-char variable names as well.
+variable-rgx=[a-z_][a-z0-9_]{0,30}$

third_party/pyjson5/src/run

@@ -3,16 +3,11 @@
 from __future__ import print_function
 
 import argparse
-import os
 import subprocess
 import sys
 
 
-is_python3 = bool(sys.version_info.major == 3)
-has_python34 = False
 verbose = False
-repo_dir = os.path.abspath(os.path.dirname(__file__))
-path_to_cov = os.path.join(repo_dir, 'tools', 'cov.py')
 
 
 def call(*args, **kwargs):
@@ -25,8 +20,6 @@ def call(*args, **kwargs):
 
 def main(argv):
     parser = argparse.ArgumentParser(prog='run')
-    parser.add_argument('--no3', action='store_true',
-                        help='Do not run the tests under Python 3.')
     parser.add_argument('-v', '--verbose', action='store_true')
     subps = parser.add_subparsers()
 
@@ -37,16 +30,9 @@ def main(argv):
     subp.set_defaults(func=run_clean)
 
     subp = subps.add_parser('coverage',
-                            help='Run the tests and report code coverage.')
+                            help='Run tests and report code coverage.')
     subp.set_defaults(func=run_coverage)
 
-    subp = subps.add_parser('develop',
-                             help='Install a symlinked package locally.')
-    subp.set_defaults(func=run_develop)
-    subp.add_argument('--system', action='store_true',
-                      help=('Install to the system site-package dir '
-                            'rather than the user\'s (requires root).'))
-
     subp = subps.add_parser('format',
                             help='Reformat the source code.')
     subp.set_defaults(func=run_format)
@@ -77,34 +63,31 @@ def main(argv):
     global verbose
     if args.verbose:
         verbose = True
-    global has_python34
-    if not args.no3:
-        try:
-            ver = subprocess.check_output(['python3', '--version'])
-            has_python34 = ver.split()[1] >= '3.4'
-        except:
-            pass
     args.func(args)
 
 
 def run_build(args):
+    del args
     call([sys.executable, 'setup.py', 'build', '--quiet'])
 
 
 def run_clean(args):
+    del args
     call(['git', 'clean', '-fxd'])
 
 
 def run_coverage(args):
-    call(['typ', '-c', 'json5'])
-
-
-def run_develop(args):
-    call([sys.executable, 'setup.py', 'develop'])
+    del args
+    call(['python', '-m', 'coverage', 'run', '-m', 'unittest',
+          'discover', '-p', '*_test.py'])
+    call(['python3', '-m', 'coverage', 'run', '--append', '-m', 'unittest',
+          'discover', '-p', '*_test.py'])
+    call([sys.executable, '-m', 'coverage', 'report', '--show-missing'])
 
 
 def run_format(args):
-    call('autopep8 --in-place *.py */*.py */*/*.py', shell=True)
+    del args
+    call('autopep8 --in-place *.py */*.py', shell=True)
 
 
 def run_help(args):
@@ -122,12 +105,14 @@ def run_install(args):
 
 
 def run_lint(args):
-    call('pylint --rcfile=pylintrc */*.py */*/*.py', shell=True)
-    call('pep8 *.py */*.py */*/*.py', shell=True)
+    del args
+    call('pylint --rcfile=pylintrc */*.py', shell=True)
 
 
 def run_tests(args):
-    call(['typ', 'json5'])
+    del args
+    call([sys.executable, '-m', 'unittest', 'discover',
+          '-p', '*_test.py'])
 
 
 if __name__ == '__main__':

third_party/pyjson5/src/setup.cfg

+[metadata]
+license_files = LICENSE.txt
+
 [bdist_wheel]
 universal=1

third_party/pyjson5/src/setup.py

@@ -23,15 +23,13 @@ if here not in sys.path:
 
 import json5
 
-with open(os.path.join(here, 'README.rst')) as fp:
-    readme = fp.read().strip()
+with open(os.path.join(here, 'README.md')) as fp:
+    long_description = fp.read()
 
-readme_lines = readme.splitlines()
 
 setup(
     name='json5',
-    packages=find_packages(),
-    package_data={'': ['../README.rst']},
+    packages=find_packages(exclude=['tests']),
     entry_points={
         'console_scripts': [
             'pyjson5=json5.tool:main',
@@ -39,11 +37,17 @@ setup(
     },
     install_requires=[
     ],
+    extras_require={
+        'dev': [
+            'hypothesis'
+        ]
+    },
     version=json5.VERSION,
     author='Dirk Pranke',
     author_email='dpranke@chromium.org',
-    description=readme_lines[3],
-    long_description=('\n' + '\n'.join(readme_lines)),
+    description=long_description.splitlines()[2],
+    long_description=long_description,
+    long_description_content_type='text/markdown',
     url='https://github.com/dpranke/pyjson5',
     license='Apache',
     classifiers=[

third_party/pyjson5/src/tests/host_fake.py

+# Copyright 2014 Dirk Pranke. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import io
+import sys
+
+if sys.version_info[0] < 3:
+    # pylint: disable=redefined-builtin, invalid-name
+    str = unicode
+
+
+class FakeHost(object):
+    # "too many instance attributes" pylint: disable=R0902
+    # "redefining built-in" pylint: disable=W0622
+    # "unused arg" pylint: disable=W0613
+
+    python_interpreter = 'python'
+
+    def __init__(self):
+        self.stdin = io.StringIO()
+        self.stdout = io.StringIO()
+        self.stderr = io.StringIO()
+        self.platform = 'linux2'
+        self.sep = '/'
+        self.dirs = set([])
+        self.files = {}
+        self.written_files = {}
+        self.last_tmpdir = None
+        self.current_tmpno = 0
+        self.cwd = '/tmp'
+
+    def abspath(self, *comps):
+        relpath = self.join(*comps)
+        if relpath.startswith('/'):
+            return relpath
+        return self.join(self.cwd, relpath)
+
+    def chdir(self, *comps):  # pragma: no cover
+        path = self.join(*comps)
+        if not path.startswith('/'):
+            path = self.join(self.cwd, path)
+        self.cwd = path
+
+    def dirname(self, path):
+        return '/'.join(path.split('/')[:-1])
+
+    def getcwd(self):
+        return self.cwd
+
+    def join(self, *comps):  # pragma: no cover
+        p = ''
+        for c in comps:
+            if c in ('', '.'):
+                continue
+            elif c.startswith('/'):
+                p = c
+            elif p:
+                p += '/' + c
+            else:
+                p = c
+
+        # Handle ./
+        p = p.replace('/./', '/')
+
+        # Handle ../
+        while '/..' in p:
+            comps = p.split('/')
+            idx = comps.index('..')
+            comps = comps[:idx-1] + comps[idx+1:]
+            p = '/'.join(comps)
+        return p
+
+    def maybe_mkdir(self, *comps):  # pragma: no cover
+        path = self.abspath(self.join(*comps))
+        if path not in self.dirs:
+            self.dirs.add(path)
+
+    def mkdtemp(self, suffix='', prefix='tmp', dir=None, **_kwargs):
+        if dir is None:
+            dir = self.sep + '__im_tmp'
+        curno = self.current_tmpno
+        self.current_tmpno += 1
+        self.last_tmpdir = self.join(dir, '%s_%u_%s' % (prefix, curno, suffix))
+        self.dirs.add(self.last_tmpdir)
+        return self.last_tmpdir
+
+    def print_(self, msg=u'', end=u'\n', stream=None):
+        stream = stream or self.stdout
+        stream.write(str(msg) + str(end))
+        stream.flush()
+
+    def read_text_file(self, *comps):
+        return self._read(comps)
+
+    def _read(self, comps):
+        return self.files[self.abspath(*comps)]
+
+    def remove(self, *comps):
+        path = self.abspath(*comps)
+        self.files[path] = None
+        self.written_files[path] = None
+
+    def rmtree(self, *comps):
+        path = self.abspath(*comps)
+        for f in self.files:
+            if f.startswith(path):
+                self.remove(f)
+        self.dirs.remove(path)
+
+    def write_text_file(self, path, contents):
+        self._write(path, contents)
+
+    def _write(self, path, contents):
+        full_path = self.abspath(path)
+        self.maybe_mkdir(self.dirname(full_path))
+        self.files[full_path] = contents
+        self.written_files[full_path] = contents

third_party/pyjson5/src/tests/host_test.py

+# Copyright 2019 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import io
+import unittest
+
+from json5.host import Host
+
+class HostTest(unittest.TestCase):
+    maxDiff = None
+
+    def test_directory_and_file_operations(self):
+        h = Host()
+        orig_cwd = h.getcwd()
+
+        try:
+            d = h.mkdtemp()
+            h.chdir(d)
+            h.write_text_file('foo', 'bar')
+            contents = h.read_text_file('foo')
+            self.assertEqual(contents, 'bar')
+            h.chdir('..')
+            h.rmtree(d)
+        finally:
+            h.chdir(orig_cwd)
+
+    def test_print(self):
+        s = io.StringIO()
+        h = Host()
+        h.print_('hello, world', stream=s)
+        self.assertEqual('hello, world\n', s.getvalue())
+
+
+if __name__ == '__main__':  # pragma: no cover
+    unittest.main()

third_party/pyjson5/src/tests/tool_test.py

+# Copyright 2017 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import sys
+import unittest
+
+import json5
+import json5.tool
+
+from .host_fake import FakeHost
+
+
+if sys.version_info[0] < 3:
+    # pylint: disable=redefined-builtin, invalid-name
+    str = unicode
+
+
+class CheckMixin(object):
+    def _write_files(self, host, files):
+        for path, contents in list(files.items()):
+            host.write_text_file(path, contents)
+
+    def check_cmd(self, args, stdin=None, files=None,
+                  returncode=None, out=None, err=None):
+        host = self._host()
+        orig_wd, tmpdir = None, None
+        try:
+            orig_wd = host.getcwd()
+            tmpdir = host.mkdtemp()
+            host.chdir(tmpdir)
+            if files:
+                self._write_files(host, files)
+            rv = self._call(host, args, stdin, returncode, out, err)
+            actual_ret, actual_out, actual_err = rv
+        finally:
+            if tmpdir:
+                host.rmtree(tmpdir)
+            if orig_wd:
+                host.chdir(orig_wd)
+
+        return actual_ret, actual_out, actual_err
+
+
+class UnitTestMixin(object):
+    def _host(self):
+        return FakeHost()
+
+    def _call(self, host, args, stdin=None,
+              returncode=None, out=None, err=None):
+        if stdin is not None:
+            host.stdin.write(str(stdin))
+            host.stdin.seek(0)
+        actual_ret = json5.tool.main(args, host)
+        actual_out = host.stdout.getvalue()
+        actual_err = host.stderr.getvalue()
+        if returncode is not None:
+            self.assertEqual(returncode, actual_ret)
+        if out is not None:
+            self.assertEqual(out, actual_out)
+        if err is not None:
+            self.assertEqual(err, actual_err)
+        return actual_ret, actual_out, actual_err
+
+
+class ToolTest(UnitTestMixin, CheckMixin, unittest.TestCase):
+    maxDiff = None
+
+    def test_help(self):
+        self.check_cmd(['--help'], returncode=0)
+
+    def test_inline_expression(self):
+        self.check_cmd(['-c', '{foo: 1}'], returncode=0,
+                       out=u'{\n    foo: 1,\n}\n')
+
+    def test_indent(self):
+        self.check_cmd(['--indent=None', '-c', '[1]'], returncode=0,
+                       out=u'[1]\n')
+        self.check_cmd(['--indent=2', '-c', '[1]'], returncode=0,
+                       out=u'[\n  1,\n]\n')
+        self.check_cmd(['--indent=  ', '-c', '[1]'], returncode=0,
+                       out=u'[\n  1,\n]\n')
+
+    def test_as_json(self):
+        self.check_cmd(['--as-json', '-c', '{foo: 1}'], returncode=0,
+                       out=u'{\n    "foo": 1\n}\n')
+
+    def test_quote_keys(self):
+        self.check_cmd(['--quote-keys', '-c', '{foo: 1}'], returncode=0,
+                       out=u'{\n    "foo": 1,\n}\n')
+
+    def test_no_quote_keys(self):
+        self.check_cmd(['--no-quote-keys', '-c', '{foo: 1}'], returncode=0,
+                       out=u'{\n    foo: 1,\n}\n')
+
+    def test_keys_are_quoted_by_default(self):
+        self.check_cmd(['-c', '{foo: 1}'], returncode=0,
+                       out=u'{\n    foo: 1,\n}\n')
+
+    def test_read_command(self):
+        self.check_cmd(['-c', '"foo"'], returncode=0, out=u'"foo"\n')
+
+    def test_read_from_stdin(self):
+        self.check_cmd([], stdin='"foo"\n', returncode=0, out=u'"foo"\n')
+
+    def test_read_from_a_file(self):
+        files = {
+            'foo.json5': '"foo"\n',
+        }
+        self.check_cmd(['foo.json5'], files=files, returncode=0, out=u'"foo"\n')
+
+    def test_trailing_commas(self):
+        self.check_cmd(['--trailing-commas', '-c', '{foo: 1}'], returncode=0,
+                       out=u'{\n    foo: 1,\n}\n')
+
+    def test_no_trailing_commas(self):
+        self.check_cmd(['--no-trailing-commas', '-c', '{foo: 1}'], returncode=0,
+                       out=u'{\n    foo: 1\n}\n')
+
+    def test_trailing_commas_are_there_by_default(self):
+        self.check_cmd(['-c', '{foo: 1}'], returncode=0,
+                       out=u'{\n    foo: 1,\n}\n')
+
+    def test_unknown_switch(self):
+        self.check_cmd(['--unknown-switch'], returncode=2,
+                       err=u'json5: error: unrecognized arguments: '
+                       '--unknown-switch\n\n')
+
+    def test_version(self):
+        self.check_cmd(['--version'], returncode=0,
+                       out=str(json5.VERSION) + '\n')
+
+
+if __name__ == '__main__':  # pragma: no cover
+    unittest.main()