Implement mb - a meta-build wrapper for bots to use in the GYP->GN migration.

This CL implements a new tool called mb (//tools/mb) that wraps both GYP
and GN to implement the 'gen' and 'analyze' steps on the bots. It
serves three purposes:

1. To move the decision to use GYP or GN to generate the build files to
   a configuration stored in the repo (and to hide it from the bots).
2. To move which flags are used to a configuration stored in the repo,
   so that we can version the flags alongside the code, and change the
   flags without needing to change things in the build repo.
3. To consolidate a list of all of the different build configs we need
   to support and verify for the GYP->GN migration.

For more details on the above, see //tools/md/README.md and the linked
docs.

R=scottmg@chromium.org, brettw@chromium.org
BUG=466436

Review URL: https://codereview.chromium.org/1062613004

Cr-Commit-Position: refs/heads/master@{#324231}

tools/mb/README.md

+# MB - The Meta-Build wrapper
+
+MB is a simple wrapper intended to provide a uniform interface to either
+GYP or GN, such that users and bots can call one script and not need to
+worry about whether a given bot is meant to use GN or GYP.
+
+It supports two main functions:
+
+1. "gen" - the main `gyp_chromium` / `gn gen` invocation that generates the
+   Ninja files needed for the build.
+
+2. "analyze" - the step that takes a list of modified files and a list of
+   desired targets and reports which targets will need to be rebuilt.
+
+We also use MB as a forcing function to collect all of the different 
+build configurations that we actually support for Chromium builds into
+one place, in `//tools/mb/mb_config.pyl`.
+
+For more information, see:
+
+* [the user guide](docs/user_guide.md)
+* [the design spec](docs/design_spec.md)

tools/mb/docs/design_spec.md

+# The MB (Meta-Build wrapper) design spec
+
+[TOC]
+
+## Intro
+
+MB is intended to address two major aspects of the GYP -> GN transition
+for Chromium:
+
+1. "bot toggling" - make it so that we can easily flip a given bot
+   back and forth between GN and GYP.
+
+2. "bot configuration" - provide a single source of truth for all of
+   the different configurations (os/arch/`gyp_define` combinations) of
+   Chromium that are supported.
+
+MB must handle at least the `gen` and `analyze` steps on the bots, i.e.,
+we need to wrap both the `gyp_chromium` invocation to generate the
+Ninja files, and the `analyze` step that takes a list of modified files
+and a list of targets to build and returns which targets are affected by
+the files.
+
+## Design
+
+MB is intended to be as simple as possible, and to defer as much work as
+possible to GN or GYP. It should live as a very simple Python wrapper
+that offers little in the way of surprises.
+
+### Command line
+
+It is structured as a single binary that supports a list of subcommands:
+
+* `mb gen -c linux_rel_bot //out/Release`
+* `mb analyze -m tryserver.chromium.linux -b linux_rel /tmp/input.json /tmp/output.json`
+
+### Configurations
+
+`mb` looks in the `//tools/mb/mb_config.pyl` config file to determine whether
+to use GYP or GN for a particular build directory, and what set of flags
+(`GYP_DEFINES` or `gn args`) to use.
+
+A config can either be specified directly (useful for testing) or by specifying
+the master name and builder name (useful on the bots so that they do not need
+to specify a config directly and can be hidden from the details).
+
+See the [user guide](user_guide.md#mb_config.pyl) for details.
+
+### Handling the analyze step
+
+The interface to `mb analyze` is described in the
+[user\_guide](user_guide.md#mb_analyze).
+
+Since the interface basically mirrors the way the "analyze" step on the bots
+invokes gyp\_chromium today, when the config is found to be a gyp config,
+the arguments are passed straight through.
+
+It implements the equivalent functionality in GN by calling `'gn refs
+[list of files] --type=executable --all --as=output` and filtering the
+output to match the list of targets.
+
+## Detailed Design Requirements and Rationale
+
+This section is collection of semi-organized notes on why MB is the way
+it is ...
+
+### in-tree or out-of-tree
+
+The first issue is whether or not this should exist as a script in
+Chromium at all; an alternative would be to simply change the bot
+configurations to know whether to use GYP or GN, and which flags to
+pass.
+
+That would certainly work, but experience over the past two years
+suggests a few things:
+
+  * we should push as much logic as we can into the source repositories
+    so that they can be versioned and changed atomically with changes to
+    the product code; having to coordinate changes between src/ and
+    build/ is at best annoying and can lead to weird errors.
+  * the infra team would really like to move to providing
+    product-independent services (i.e., not have to do one thing for
+    Chromium, another for NaCl, a third for V8, etc.).
+  * we found that during the SVN->GIT migration the ability to flip bot
+    configurations between the two via changes to a file in chromium
+    was very useful.
+
+All of this suggests that the interface between bots and Chromium should
+be a simple one, hiding as much of the chromium logic as possible.
+
+### Why not have MB be smarter about de-duping flags?
+
+This just adds complexity to the MB implementation, and duplicates logic
+that GYP and GN already have to support anyway; in particular, it might
+require MB to know how to parse GYP and GN values. The belief is that
+if MB does *not* do this, it will lead to fewer surprises.
+
+It will not be hard to change this if need be.
+
+### Integration w/ gclient runhooks
+
+On the bots, we will disable `gyp_chromium` as part of runhooks (using
+`GYP_CHROMIUM_NO_ACTION=1`), so that mb shows up as a separate step.
+
+At the moment, we expect most developers to either continue to use
+`gyp_chromium` in runhooks or to disable at as above if they have no
+use for GYP at all. We may revisit how this works once we encourage more
+people to use GN full-time (i.e., we might take `gyp_chromium` out of
+runhooks altogether).
+
+### Config per flag set or config per (os/arch/flag set)?
+
+Currently, mb_config.pyl does not specify the host_os, target_os, host_cpu, or
+target_cpu values for every config that Chromium runs on, it only specifies
+them for when the values need to be explicitly set on the command line.
+
+Instead, we have one config per unique combination of flags only.
+
+In other words, rather than having `linux_rel_bot`, `win_rel_bot`, and
+`mac_rel_bot`, we just have `rel_bot`.
+
+This design allows us to determine easily all of the different sets
+of flags that we need to support, but *not* which flags are used on which
+host/target combinations.
+
+It may be that we should really track the latter. Doing so is just a 
+config file change, however.
+
+### Non-goals
+
+* MB is not intended to replace direct invocation of GN or GYP for
+  complicated build scenarios (aka ChromeOS), where multiple flags need
+  to be set to user-defined paths for specific toolchains (e.g., where
+  ChromeOS needs to specify specific board types and compilers).
+
+* MB is not intended at this time to be something developers use frequently,
+  or to add a lot of features to. We hope to be able to get rid of it once
+  the GYP->GN migration is done, and so we should not add things for
+  developers that can't easily be added to GN itself.
+
+* MB is not intended to replace the 
+  [CR tool](https://code.google.com/p/chromium/wiki/CRUserManual). Not
+  only is it only intended to replace the gyp\_chromium part of `'gclient
+  runhooks'`, it is not really meant as a developer-facing tool.
+
+### Open issues
+
+* Some common flags (goma\_dir being the obvious one) may need to be
+  specified via the user, and it's unclear how to integrate this with
+  the concept of build\_configs.
+  
+  Right now, MB has hard-coded support for a few flags (i.e., you can
+  pass the --goma-dir flag, and it will know to expand "${goma\_dir}" in
+  the string before calling out to the tool. We may want to generalize
+  this to a common key/value approach (perhaps then meeting the
+  ChromeOS non-goal, above), or we may want to keep this very strictly
+  limited for simplicity.

tools/mb/docs/user_guide.md

+# The MB (Meta-Build wrapper) user guide
+
+[TOC]
+
+## Introduction
+
+`mb` is a simple python wrapper around the GYP and GN meta-build tools to
+be used as part of the GYP->GN migration.
+
+It is intended to be used by bots to make it easier to manage the configuration
+each bot builds (i.e., the configurations can be changed from chromium
+commits), and to consolidate the list of all of the various configurations
+that Chromium is built in.
+
+Ideally this tool will no longer be needed after the migration is complete.
+
+## `mb gen`
+
+`mb gen` is responsible for generating the Ninja files by invoking either GYP
+or GN as appropriate. It takes arguments to specify a build config and
+a directory, then runs GYP or GN as appropriate:
+
+```
+% mb gen -m tryserver.chromium.linux -b linux_rel //out/Release
+% mb gen -c linux_rel_trybot //out/Release
+```
+
+Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags
+must be specified so that `mb` can figure out which config to use.
+
+The path must be a GN-style path relative to the checkout root (as above).
+
+If gen ends up using GYP, the path must have a valid GYP configuration as the
+last component of the path (i.e., specify `//out/Release_x64`, not `//out`).
+
+## `mb analyze`
+
+`mb analyze` is reponsible for determining what targets are affected by
+a list of files (e.g., the list of files in a patch on a trybot):
+
+```
+mb analyze -c chromium_linux_rel //out/Release input.json output.json
+```
+
+Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags
+must be specified so that `mb` can figure out which config to use.
+
+
+The first positional argument is the path to the build directory.
+
+The second positional argument is a path to a JSON file containing a single
+object with two fields:
+
+  * `files`: an array of the modified filenames to check (as
+    paths relative to the checkout root).
+  * `targets`: an array of the unqualified target names to check.
+
+The third positional argument is a path where the mb will write the result,
+also as a JSON object. This object may contain the following fields:
+
+  * `error`: this should only be present if something failed.
+  * `targets`: the subset of the input `targets` that depend on the
+    input `files`.
+  * `build_targets`: the minimal subset of targets needed to build all
+    of `targets` that were affected.
+  * `status`: one of three strings:
+    * `"Found dependency"` (build the `build_targets`)
+    * `"No dependency"` (i.e., no build needed)
+    * `"Found dependency (all)"` (build everything, in which case
+      `targets` and `build_targets` are not returned).
+
+## `mb help`
+
+Produces help output on the other subcommands
+
+## `mb lookup`
+
+Prints what command will be run by `mb gen` (like `mb gen -n` but does
+not require you to specify a path).
+
+## `mb validate`
+
+Does internal checking to make sure the config file is syntactically
+valid and that all of the entries are used properly. It does not validate
+that the flags make sense, or that the builder names are legal or
+comprehensive, but it does complain about configs and mixins that aren't
+used.
+
+This is mostly useful as a presubmit check and for verifying changes to
+the config file.
+
+## mb_config.pyl
+
+The `mb_config.pyl` config file is intended to enumerate all of the
+supported build configurations for Chromium. Generally speaking, you
+should never need to (or want to) build a configuration that isn't
+listed here, and so by using the configs in this file you can avoid
+having to juggle long lists of GYP_DEFINES and gn args by hand.
+
+`mb_config.pyl` is structured as a file containing a single PYthon Literal
+expression: a dictionary with three main keys, `masters`, `configs` and
+`mixins`.
+
+The `masters` key contains a nested series of dicts containing mappings
+of master -> builder -> config . This allows us to isolate the buildbot
+recipes from the actual details of the configs.
+
+The `configs` key points to a dictionary of named build
+configurations.
+
+There should be an key in this dict for every supported configuration
+of Chromium, meaning every configuration we have a bot for, and every
+configuration commonly used by develpers but that we may not have a bot
+for.
+
+The value of each key is a list of "mixins" that will define what that
+build_config does. Each item in the list must be an entry in the dictionary
+value of the `mixins` key.
+
+Each mixin value is itself a dictionary that contains one or more of the
+following keys:
+
+  * `gyp_configs`: a list of the configurations to build, e.g.,
+    ['Release', 'Release_x64'].
+  * `gyp_defines`: a string containing a list of GYP_DEFINES.
+  * `gn_args`: a string containing a list of values passed to gn --args.
+  * `mixins`: a list of other mixins that should be included.
+  * `type`: a string with either the value `gyp` or `gn`;
+    setting this indicates which meta-build tool to use.
+
+When `mb gen` or `mb analyze` executes, it takes a config name, looks it
+up in the 'configs' dict, and then does a left-to-right expansion of the
+mixins; gyp_defines and gn_args values are concatenated, and type and
+gyp_configs values override each other.
+
+For example, if you had:
+
+```
+{
+  'configs`: {
+    'linux_release_trybot': ['gyp_release', 'trybot'],
+    'gn_shared_debug': None,
+  }
+  'mixins': {
+    'bot': {
+      'gyp_defines': 'use_goma=1 dcheck_always_on=0',
+      'gn_args': 'use_goma=true dcheck_always_on=false',
+    },
+    'debug': {
+      'gn_args': 'is_debug=true',
+    },
+    'gn': {'type': 'gn'},
+    'gyp_release': {
+      'gyp_config': 'Release'
+      'mixins': ['release'],
+      'type': 'gyp',
+    },
+    'release': {
+      'gn_args': 'is_debug=false',
+    }
+    'shared': {
+      'gn_args': 'is_component_build=true',
+      'gyp_defines': 'component=shared_library',
+    },
+    'trybot': {
+      'gyp_defines': 'dcheck_always_on=1',
+      'gn_args': 'dcheck_always_on=true',
+    }
+  }
+}
+
+and you ran `mb gen -c linux_release_trybot //out/Release`, it would
+translate into a call to `gyp_chromium -G Release` with `GYP_DEFINES` set to
+`"use_goma=true dcheck_always_on=false dcheck_always_on=true"`.
+
+(From that you can see that mb is intentionally dumb and does not
+attempt to de-dup the flags, it lets gyp do that).
+
+## Debugging (-v/--verbose and -n/--dry-run)
+
+By design, MB should be simple enough that very little can go wrong.
+
+The most obvious issue is that you might see different commands being
+run than you expect; running `'mb -v'` will print what it's doing and
+run the commands; `'mb -n'` will print what it will do but *not* run
+the commands.
+
+If you hit weirder things than that, add some print statements to the
+python script, send a question to gn-dev@chromium.org, or
+[file a bug](https://crbug.com/new) with the label
+'mb' and cc: dpranke@chromium.org.
+
+

tools/mb/mb

+#!/usr/bin/env bash
+# Copyright 2015 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+base_dir=$(dirname "$0")
+
+PYTHONDONTWRITEBYTECODE=1 exec python "$base_dir/mb.py" "$@"

tools/mb/mb.bat

+@echo off
+setlocal
+:: This is required with cygwin only.
+PATH=%~dp0;%PATH%
+set PYTHONDONTWRITEBYTECODE=1
+call python "%~dp0mb.py" %*

tools/mb/mb_config.pyl

+# Copyright 2015 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+{
+  # This is the list of configs that you can pass to mb; each config
+  # represents a particular combination of GYP_DEFINES/gn args that
+  # we must support. A given config *may* be platform-specific but
+  # is not necessarily so (i.e., we might have mac, win, and linux
+  # bots all using the 'gn_release_bot' config).
+  'configs': {
+    'android_gn_release_bot': ['android', 'gn', 'release_bot'],
+    'android_gn_debug_bot': ['android', 'gn', 'debug_bot'],
+    'android_gn_release_trybot': ['android', 'gn', 'release_trybot'],
+    'chromeos_gn_debug_bot': ['chromeos', 'gn', 'debug_bot'],
+    'chromeos_gn_release_bot': ['chromeos', 'gn', 'release_bot'],
+    'chromeos_gn_release_trybot': ['chromeos', 'gn', 'release_trybot'],
+    'dev_gn_debug': ['gn', 'debug', 'shared', 'full_symbols'],
+    'dev_gyp_debug': ['gyp', 'debug', 'shared', 'full_symbols'],
+    'dev_gn_release': ['gn', 'release', 'shared'],
+    'dev_gyp_release': ['gyp', 'release', 'shared'],
+    'gn_release_bot': ['gn', 'release_bot'],
+    'gn_release_bot_x86': ['gn', 'release_bot', 'x86'],
+    'gn_release_trybot': ['gn', 'release_trybot'],
+    'gn_debug_bot': ['gn', 'debug_bot'],
+    'gyp_release_bot': ['gyp', 'release_bot'],
+  },
+  
+  # This is a list of configs that do not actually exist on any bot
+  # but are used so commonly by devs that we must support them.
+  'common_dev_configs': [
+    'dev_gn_debug',
+    'dev_gn_release',
+    'dev_gyp_debug',
+    'dev_gyp_release',
+  ],
+  
+  # This is a list of configs that some private (not publicly accessible)
+  # bot somewhere uses and that we must support. Ideally we should actually
+  # have a bot for each of these on the public waterfall. Each config should
+  # at least have a contact listed.
+  'private_configs': [
+  ],
+
+  # This is a list of configs that are not commonly used by that we should
+  # make some effort to support, but if it breaks that is not the end of
+  # the world. Each config should have a contact listed, and we expect the
+  # contact to be on the hook for fixing the config.
+  'unsupported_configs': [
+  ],
+  
+  # This is a dict mapping a given 'mixin' name to a dict of settings that
+  # mb should use. See //tools/mb/docs/user_guide.md for more information.
+  'mixins': {
+    'android': {
+      'gn_args': 'target_os="android"',
+      'gyp_defines': 'OS=android',
+    },
+
+    'chromeos': {
+      'gn_args': 'target_os="chromeos"',
+      'gyp_defines': 'chromeos=1',
+    },
+
+    'dcheck_always_on': {
+      'gn_args': 'dcheck_always_on=true',
+      'gyp_defines': 'dcheck_always_on=1',
+    },
+
+    'debug': {
+      'gn_args': 'is_debug=true', 
+      'gyp_config': 'Debug',
+    },
+
+    'debug_bot': {'mixins': ['debug', 'shared', 'minimal_symbols', 'goma']},
+
+    'full_symbols': {
+      'gn_args': 'symbol_level=2',
+      'gyp_defines': 'fastbuild=0',
+    },
+
+    'gn': {'type': 'gn'},
+
+    'goma': {
+      'gn_args': 'use_goma=true goma_dir="$(goma_dir)"',
+      'gyp_defines': 'goma=1 gomadir=$(goma_dir)',
+    },
+
+    'gyp': {'type': 'gyp'},
+    
+    'minimal_symbols': {
+      'gn_args': 'symbol_level=1',
+      'gyp_defines': 'fastbuild=1',
+    },
+
+    'release': {
+      'gn_args': 'is_debug=false',
+      'gyp_config': 'Release',
+    },
+
+    'release_bot': {
+      'mixins': ['release', 'static', 'minimal_symbols', 'goma'],
+    },
+
+    'release_trybot': {
+      'mixins': ['release_bot', 'dcheck_always_on', 'goma']
+    },
+    
+    'shared': {
+      'gn_args': 'is_component_build=true',
+      'gyp_defines': 'component=shared_library',
+    },
+
+    'static': {
+      'gn_args': 'is_component_build=false',
+      'gyp_defines': 'component=static_library',
+    },
+
+    'x86': {
+      'gn_args': 'target_cpu="x86"',
+      'gyp_args': 'target_arch=ia32',
+    },
+  },
+
+
+  # This is a map of buildbot master names -> buildbot builder names ->
+  # config names (where each config name is a key in the 'configs' dict,
+  # above). mb uses this dict to look up which config to use for a given bot.
+  # TODO(dpranke): add in remaining bots on the waterfalls.
+  'masters': {
+    'chromium.chromeos': {
+      'Linux ChromiumOS GN': 'chromeos_gn_release_bot',
+    },
+    'chromium.linux': {
+      'Android GN': 'android_gn_release_bot',
+      'Linux Builder': 'gyp_release_bot',
+      'Linux GN': 'gn_release_bot',
+      'Linux GN (dbg)': 'gn_debug_bot',
+      'Linux Tests': 'gyp_release_bot',
+    },
+    'chromium.mac': {
+      'Mac GN': 'gn_release_bot',
+      'Mac GN (dbg)': 'gn_debug_bot',
+    },
+    'chromium.win': {
+      'Win8 GN': 'gn_release_bot',
+      'Win8 GN (dbg)': 'gn_debug_bot',
+    },
+    'tryserver.chromium.linux': {
+      'android_chromium_gn_dbg': 'android_gn_debug_bot',
+      'android_chromium_gn_rel': 'android_gn_release_trybot',
+      'linux_chromiumos_chromium_gn_rel': 'chromeos_gn_release_trybot', 
+      'linux_chromiumos_chromium_gn_dbg': 'chromeos_gn_debug_bot',
+      'linux_chromium_gn_dbg': 'gn_debug_bot', 
+      'linux_chromium_gn_rel': 'gn_release_trybot', 
+      'linux_chromium_gn_upload_x64': 'gn_release_bot',
+      'linux_chromium_gn_upload_x86': 'gn_release_bot_x86',
+    },
+    'tryserver.chromium.mac': {
+      'mac_chromium_gn_dbg': 'gn_debug_bot',
+      'mac_chromium_gn_rel': 'gn_release_trybot',
+      'mac_chromium_gn_upload': 'gn_release_bot',
+    },
+    'tryserver.chromium.win': {
+      'win8_chromium_gn_dbg': 'gn_debug_bot',
+      'win8_chromium_gn_rel': 'gn_release_trybot',
+      'win8_chromium_gn_upload': 'gn_release_bot',
+    },
+  },
+}

tools/mb/mb_unittest.py

+# Copyright 2015 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+"""Tests for mb.py."""
+
+import sys
+import unittest
+
+import mb
+
+
+class FakeMB(mb.MetaBuildWrapper):
+  def __init__(self):
+    super(FakeMB, self).__init__()
+    self.files = {}
+    self.calls = []
+    self.out = []
+    self.err = []
+    self.chromium_src_dir = '/fake_src'
+    self.default_config = '/fake_src/tools/mb/mb_config.pyl'
+
+  def ExpandUser(self, path):
+    return '$HOME/%s' % path
+
+  def Exists(self, path):
+    return self.files.get(path) is not None
+
+  def ReadFile(self, path):
+    return self.files[path]
+
+  def WriteFile(self, path, contents):
+    self.files[path] = contents
+
+  def Call(self, cmd):
+    self.calls.append(cmd)
+    return 0, '', ''
+
+  def Print(self, *args, **kwargs):
+    sep = kwargs.get('sep', ' ')
+    end = kwargs.get('end', '\n')
+    f = kwargs.get('file', sys.stdout)
+    if f == sys.stderr:
+      self.err.append(sep.join(args) + end)
+    else:
+      self.out.append(sep.join(args) + end)
+
+class IntegrationTest(unittest.TestCase):
+  def test_validate(self):
+    # Note that this validates that the actual mb_config.pyl is valid.
+    ret = mb.main(['validate', '--quiet'])
+    self.assertEqual(ret, 0)
+
+
+TEST_CONFIG = """\
+{
+  'common_dev_configs': ['gn_debug'],
+  'configs': {
+    'gyp_rel_bot': ['gyp', 'rel', 'goma'],
+    'gn_debug': ['gn', 'debug'],
+    'private': ['gyp', 'fake_feature1'],
+    'unsupported': ['gn', 'fake_feature2'],
+  },
+  'masters': {
+    'fake_master': {
+      'fake_builder': 'gyp_rel_bot',
+    },
+  },
+  'mixins': {
+    'fake_feature1': {
+      'gn_args': 'enable_doom_melon=true',
+      'gyp_defines': 'doom_melon=1',
+    },
+    'fake_feature2': {
+      'gn_args': 'enable_doom_melon=false',
+      'gyp_defaults': 'doom_melon=0',
+    },
+    'gyp': {'type': 'gyp'},
+    'gn': {'type': 'gn'},
+    'goma': {
+      'gn_args': 'use_goma=true goma_dir="$(goma_dir)"',
+      'gyp_defines': 'goma=1 gomadir="$(goma_dir)"',
+    },
+    'rel': {
+      'gn_args': 'is_debug=false',
+      'gyp_config': 'Release',
+    },
+    'debug': {
+      'gn_args': 'is_debug=true',
+    },
+  },
+  'private_configs': ['private'],
+  'unsupported_configs': ['unsupported'],
+}
+"""
+
+
+class UnitTest(unittest.TestCase):
+  def check(self, args, files=None, cmds=None, out=None, err=None, ret=None):
+    m = FakeMB()
+    if files:
+      for path, contents in files.items():
+        m.files[path] = contents
+    m.files.setdefault(mb.default_config, TEST_CONFIG)
+    m.ParseArgs(args)
+    actual_ret = m.args.func()
+    if ret is not None:
+      self.assertEqual(actual_ret, ret)
+    if out is not None:
+      self.assertEqual(m.out, out)
+    if err is not None:
+      self.assertEqual(m.err, err)
+    if cmds is not None:
+      self.assertEqual(m.cmds, cmds)
+
+  def test_analyze(self):
+    files = {'/tmp/in.json': '{"files": [], "targets": []}'}
+    self.check(['analyze', '-c', 'gn_debug', '//out/Default',
+                '/tmp/in.json', '/tmp/out.json'],
+               files=files, ret=0)
+    self.check(['analyze', '-c', 'gyp_rel_bot', '//out/Release',
+                '/tmp/in.json', '/tmp/out.json'],
+               ret=0)
+
+  def test_gen(self):
+    self.check(['gen', '-c', 'gn_debug', '//out/Default'], ret=0)
+    self.check(['gen', '-c', 'gyp_rel_bot', '//out/Release'], ret=0)
+
+  def test_help(self):
+    self.assertRaises(SystemExit, self.check, ['-h'])
+    self.assertRaises(SystemExit, self.check, ['help'])
+    self.assertRaises(SystemExit, self.check, ['help', 'gen'])
+
+  def test_lookup(self):
+    self.check(['lookup', '-c', 'gn_debug'], ret=0)
+
+  def test_validate(self):
+    self.check(['validate'], ret=0)
+
+
+if __name__ == '__main__':
+  unittest.main()