Commit 74502493 authored by Ben Pastene's avatar Ben Pastene Committed by Chromium LUCI CQ

Replace all uses of the word 'master' with 'builder_group' in //tools/mb

This removes every reference but the "--master/-m" cmd-line arg and the
"masters" mb_config.pyl key, which will be removed in a follow-up once
all users of mb.py (ie: recipes) have switched over.

"builder_group" is also the term we're using when replacing "master" in
recipe code: crbug.com/1109276. So we should conform on using that term
going forward.

Bug: 1117773
Change-Id: I10a3c7a4bc5fd5146bc88d670564a745cb3fce2a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2605225
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: default avatarDirk Pranke <dpranke@google.com>
Cr-Commit-Position: refs/heads/master@{#840641}
parent ce98df46
...@@ -49,8 +49,8 @@ If no matching bot config file is found, `mb` looks in the ...@@ -49,8 +49,8 @@ If no matching bot config file is found, `mb` looks in the
(`gn args`) to use. (`gn args`) to use.
A config can either be specified directly (useful for testing) or by specifying A config can either be specified directly (useful for testing) or by specifying
the master name and builder name (useful on the bots so that they do not need the builder_group name and builder name (useful on the bots so that they do not
to specify a config directly and can be hidden from the details). need to specify a config directly and can be hidden from the details).
See the [user guide](user_guide.md#mb_config.pyl) for details. See the [user guide](user_guide.md#mb_config.pyl) for details.
......
...@@ -27,7 +27,7 @@ a list of files (e.g., the list of files in a patch on a trybot): ...@@ -27,7 +27,7 @@ a list of files (e.g., the list of files in a patch on a trybot):
mb analyze -c chromium_linux_rel //out/Release input.json output.json mb analyze -c chromium_linux_rel //out/Release input.json output.json
``` ```
Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags Either the `-c/--config` flag or the `--builder-group` and `-b/--builder` flags
must be specified so that `mb` can figure out which config to use. must be specified so that `mb` can figure out which config to use.
The first positional argument must be a GN-style "source-absolute" path The first positional argument must be a GN-style "source-absolute" path
...@@ -90,7 +90,7 @@ differences can be subtle. We won't even go into how the `targets` and ...@@ -90,7 +90,7 @@ differences can be subtle. We won't even go into how the `targets` and
`build_targets` differ from each other or from `compile_targets` and `build_targets` differ from each other or from `compile_targets` and
`test_targets`. `test_targets`.
The `-b/--builder`, `-c/--config`, `-f/--config-file`, `-m/--master`, The `-b/--builder`, `-c/--config`, `-f/--config-file`, `--builder-group`,
`-q/--quiet`, and `-v/--verbose` flags work as documented for `mb gen`. `-q/--quiet`, and `-v/--verbose` flags work as documented for `mb gen`.
### mb gen ### mb gen
...@@ -104,7 +104,7 @@ specify a build config and a directory, then runs GN as appropriate: ...@@ -104,7 +104,7 @@ specify a build config and a directory, then runs GN as appropriate:
% mb gen -c linux_rel_trybot //out/Release % mb gen -c linux_rel_trybot //out/Release
``` ```
Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags Either the `-c/--config` flag or the `--builder-group` and `-b/--builder` flags
must be specified so that `mb` can figure out which config to use. The must be specified so that `mb` can figure out which config to use. The
`--phase` flag must also be used with builders that have multiple `--phase` flag must also be used with builders that have multiple
build/compile steps (and only with those builders). build/compile steps (and only with those builders).
...@@ -144,7 +144,7 @@ swarming. See below for more information on isolates and swarming. ...@@ -144,7 +144,7 @@ swarming. See below for more information on isolates and swarming.
Prints what command will be run by `mb gen` (like `mb gen -n` but does Prints what command will be run by `mb gen` (like `mb gen -n` but does
not require you to specify a path). not require you to specify a path).
The `-b/--builder`, `-c/--config`, `-f/--config-file`, `-m/--master`, The `-b/--builder`, `-c/--config`, `-f/--config-file`, `--builder-group`,
`--phase`, `-q/--quiet`, and `-v/--verbose` flags work as documented for `--phase`, `-q/--quiet`, and `-v/--verbose` flags work as documented for
`mb gen`. `mb gen`.
...@@ -176,8 +176,9 @@ like one you could trigger via `git cl try` or via CQ dry runs. Basic usage is ...@@ -176,8 +176,9 @@ like one you could trigger via `git cl try` or via CQ dry runs. Basic usage is
Your change must be uploaded to Gerrit. Local changes will not be uploaded for Your change must be uploaded to Gerrit. Local changes will not be uploaded for
you. It uses the gerrit CL associated with your given git branch. you. It uses the gerrit CL associated with your given git branch.
You still have to specify the mastername (`-m`) and buildername (`-b`) arguments. You still have to specify the builder group(`--builder-group`) and buildername
See [trybots.py](https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/chromium_tests/trybots.py) (`-b`) arguments. See
[trybots.py](https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/chromium_tests/trybots.py)
for a mapping of which bots are on which tryservers, and what those bots mirror. for a mapping of which bots are on which tryservers, and what those bots mirror.
Any trybot in `trybots.py` is supported; you can test your code on windows, for Any trybot in `trybots.py` is supported; you can test your code on windows, for
example. The tryjob will compile and run your code on windows. example. The tryjob will compile and run your code on windows.
...@@ -236,11 +237,11 @@ listed here, and so by using the configs in this file you can avoid ...@@ -236,11 +237,11 @@ listed here, and so by using the configs in this file you can avoid
having to juggle long lists of gn args by hand. having to juggle long lists of gn args by hand.
`mb_config.pyl` is structured as a file containing a single PYthon Literal `mb_config.pyl` is structured as a file containing a single PYthon Literal
expression: a dictionary with three main keys, `masters`, `configs` and expression: a dictionary with three main keys, `builder_groups`, `configs` and
`mixins`. `mixins`.
The `masters` key contains a nested series of dicts containing mappings The `builder_groups` key contains a nested series of dicts containing mappings
of master -> builder -> config . This allows us to isolate the buildbot of builder_group -> builder -> config . This allows us to isolate the builder
recipes from the actual details of the configs. The config should either recipes from the actual details of the configs. The config should either
be a single string value representing a key in the `configs` dictionary, be a single string value representing a key in the `configs` dictionary,
or a list of strings, each of which is a key in the `configs` dictionary; or a list of strings, each of which is a key in the `configs` dictionary;
......
...@@ -10,17 +10,17 @@ import os ...@@ -10,17 +10,17 @@ import os
import re import re
def GetAllConfigs(masters): def GetAllConfigs(builder_groups):
"""Build a list of all of the configs referenced by builders. """Build a list of all of the configs referenced by builders.
""" """
all_configs = {} all_configs = {}
for master in masters: for builder_group in builder_groups:
for config in masters[master].values(): for config in builder_groups[builder_group].values():
if isinstance(config, dict): if isinstance(config, dict):
for c in config.values(): for c in config.values():
all_configs[c] = master all_configs[c] = builder_group
else: else:
all_configs[config] = master all_configs[config] = builder_group
return all_configs return all_configs
...@@ -55,13 +55,13 @@ def CheckAllConfigsAndMixinsReferenced(errs, all_configs, configs, mixins): ...@@ -55,13 +55,13 @@ def CheckAllConfigsAndMixinsReferenced(errs, all_configs, configs, mixins):
return errs return errs
def EnsureNoProprietaryMixins(errs, masters, configs, mixins): def EnsureNoProprietaryMixins(errs, builder_groups, configs, mixins):
"""If we're checking the Chromium config, check that the 'chromium' bots """If we're checking the Chromium config, check that the 'chromium' bots
which build public artifacts do not include the chrome_with_codecs mixin. which build public artifacts do not include the chrome_with_codecs mixin.
""" """
if 'chromium' in masters: if 'chromium' in builder_groups:
for builder in masters['chromium']: for builder in builder_groups['chromium']:
config = masters['chromium'][builder] config = builder_groups['chromium'][builder]
def RecurseMixins(current_mixin): def RecurseMixins(current_mixin):
if current_mixin == 'chrome_with_codecs': if current_mixin == 'chrome_with_codecs':
...@@ -76,21 +76,21 @@ def EnsureNoProprietaryMixins(errs, masters, configs, mixins): ...@@ -76,21 +76,21 @@ def EnsureNoProprietaryMixins(errs, masters, configs, mixins):
for mixin in configs[config]: for mixin in configs[config]:
RecurseMixins(mixin) RecurseMixins(mixin)
else: else:
errs.append('Missing "chromium" master. Please update this ' errs.append('Missing "chromium" builder_group. Please update this '
'proprietary codecs check with the name of the master ' 'proprietary codecs check with the name of the builder_group '
'responsible for public build artifacts.') 'responsible for public build artifacts.')
def _GetConfigsByBuilder(masters): def _GetConfigsByBuilder(builder_groups):
"""Builds a mapping from buildername -> [config] """Builds a mapping from buildername -> [config]
Args Args
masters: the master's dict from mb_config.pyl builder_groups: the builder_group's dict from mb_config.pyl
""" """
result = collections.defaultdict(list) result = collections.defaultdict(list)
for master in masters.values(): for builder_group in builder_groups.values():
for buildername, builder in master.items(): for buildername, builder in builder_group.items():
result[buildername].append(builder) result[buildername].append(builder)
return result return result
...@@ -142,13 +142,14 @@ def CheckExpectations(mbw, jsonish_blob, expectations_dir): ...@@ -142,13 +142,14 @@ def CheckExpectations(mbw, jsonish_blob, expectations_dir):
Returns: True if expectations are up-to-date. False otherwise. Returns: True if expectations are up-to-date. False otherwise.
""" """
# Assert number of masters == number of expectation files. # Assert number of builder_groups == number of expectation files.
if len(mbw.ListDir(expectations_dir)) != len(jsonish_blob): if len(mbw.ListDir(expectations_dir)) != len(jsonish_blob):
return False return False
for master, builders in jsonish_blob.items(): for builder_group, builders in jsonish_blob.items():
if not mbw.Exists(os.path.join(expectations_dir, master + '.json')): if not mbw.Exists(os.path.join(expectations_dir, builder_group + '.json')):
return False # No expecation file for the master. return False # No expecation file for the builder_group.
expectation = mbw.ReadFile(os.path.join(expectations_dir, master + '.json')) expectation = mbw.ReadFile(os.path.join(expectations_dir,
builder_group + '.json'))
builders_json = json.dumps(builders, builders_json = json.dumps(builders,
indent=2, indent=2,
sort_keys=True, sort_keys=True,
......
...@@ -93,7 +93,7 @@ class MetaBuildWrapper(object): ...@@ -93,7 +93,7 @@ class MetaBuildWrapper(object):
self.args = argparse.Namespace() self.args = argparse.Namespace()
self.configs = {} self.configs = {}
self.public_artifact_builders = None self.public_artifact_builders = None
self.masters = {} self.builder_groups = {}
self.mixins = {} self.mixins = {}
self.isolate_exe = 'isolate.exe' if self.platform.startswith( self.isolate_exe = 'isolate.exe' if self.platform.startswith(
'win') else 'isolate' 'win') else 'isolate'
...@@ -120,7 +120,10 @@ class MetaBuildWrapper(object): ...@@ -120,7 +120,10 @@ class MetaBuildWrapper(object):
def AddCommonOptions(subp): def AddCommonOptions(subp):
group = subp.add_mutually_exclusive_group() group = subp.add_mutually_exclusive_group()
group.add_argument( group.add_argument(
'-m', '--master', help='master name to look up config from') '-m', '--builder-group',
# TODO(crbug.com/1117773): Remove the 'master' args.
'--master',
help='builder group name to look up config from')
subp.add_argument('-b', '--builder', subp.add_argument('-b', '--builder',
help='builder name to look up config from') help='builder name to look up config from')
subp.add_argument('-c', '--config', subp.add_argument('-c', '--config',
...@@ -435,8 +438,8 @@ class MetaBuildWrapper(object): ...@@ -435,8 +438,8 @@ class MetaBuildWrapper(object):
for f in self.ListDir(expectations_dir): for f in self.ListDir(expectations_dir):
self.RemoveFile(os.path.join(expectations_dir, f)) self.RemoveFile(os.path.join(expectations_dir, f))
obj = self._ToJsonish() obj = self._ToJsonish()
for master, builder in sorted(obj.items()): for builder_group, builder in sorted(obj.items()):
expectation_file = os.path.join(expectations_dir, master + '.json') expectation_file = os.path.join(expectations_dir, builder_group + '.json')
json_s = json.dumps(builder, json_s = json.dumps(builder,
indent=2, indent=2,
sort_keys=True, sort_keys=True,
...@@ -751,14 +754,14 @@ class MetaBuildWrapper(object): ...@@ -751,14 +754,14 @@ class MetaBuildWrapper(object):
"""Dumps the config file into a json-friendly expanded dict. """Dumps the config file into a json-friendly expanded dict.
Returns: Returns:
A dict with master -> builder -> all GN args mapping. A dict with builder group -> builder -> all GN args mapping.
""" """
self.ReadConfigFile(self.args.config_file) self.ReadConfigFile(self.args.config_file)
obj = {} obj = {}
for master, builders in self.masters.items(): for builder_group, builders in self.builder_groups.items():
obj[master] = {} obj[builder_group] = {}
for builder in builders: for builder in builders:
config = self.masters[master][builder] config = self.builder_groups[builder_group][builder]
if not config: if not config:
continue continue
if isinstance(config, dict): if isinstance(config, dict):
...@@ -777,7 +780,7 @@ class MetaBuildWrapper(object): ...@@ -777,7 +780,7 @@ class MetaBuildWrapper(object):
args = {'gn_args': gn_helpers.FromGNArgs(flattened_config['gn_args'])} args = {'gn_args': gn_helpers.FromGNArgs(flattened_config['gn_args'])}
if flattened_config.get('args_file'): if flattened_config.get('args_file'):
args['args_file'] = flattened_config['args_file'] args['args_file'] = flattened_config['args_file']
obj[master][builder] = args obj[builder_group][builder] = args
return obj return obj
...@@ -787,7 +790,7 @@ class MetaBuildWrapper(object): ...@@ -787,7 +790,7 @@ class MetaBuildWrapper(object):
self.ReadConfigFile(self.args.config_file) self.ReadConfigFile(self.args.config_file)
# Build a list of all of the configs referenced by builders. # Build a list of all of the configs referenced by builders.
all_configs = validation.GetAllConfigs(self.masters) all_configs = validation.GetAllConfigs(self.builder_groups)
# Check that every referenced args file or config actually exists. # Check that every referenced args file or config actually exists.
for config, loc in all_configs.items(): for config, loc in all_configs.items():
...@@ -804,11 +807,11 @@ class MetaBuildWrapper(object): ...@@ -804,11 +807,11 @@ class MetaBuildWrapper(object):
self.configs, self.mixins) self.configs, self.mixins)
if self.args.config_file == self.default_config: if self.args.config_file == self.default_config:
validation.EnsureNoProprietaryMixins(errs, self.masters, self.configs, validation.EnsureNoProprietaryMixins(errs, self.builder_groups,
self.mixins) self.configs, self.mixins)
validation.CheckDuplicateConfigs(errs, self.configs, self.mixins, validation.CheckDuplicateConfigs(errs, self.configs, self.mixins,
self.masters, FlattenConfig) self.builder_groups, FlattenConfig)
if errs: if errs:
raise MBErr(('mb config file %s has problems:\n ' % raise MBErr(('mb config file %s has problems:\n ' %
...@@ -830,7 +833,7 @@ class MetaBuildWrapper(object): ...@@ -830,7 +833,7 @@ class MetaBuildWrapper(object):
build_dir = self.args.path build_dir = self.args.path
vals = DefaultVals() vals = DefaultVals()
if self.args.builder or self.args.master or self.args.config: if self.args.builder or self.args.builder_group or self.args.config:
vals = self.Lookup() vals = self.Lookup()
# Re-run gn gen in order to ensure the config is consistent with the # Re-run gn gen in order to ensure the config is consistent with the
# build dir. # build dir.
...@@ -922,10 +925,10 @@ class MetaBuildWrapper(object): ...@@ -922,10 +925,10 @@ class MetaBuildWrapper(object):
return vals return vals
def ReadIOSBotConfig(self): def ReadIOSBotConfig(self):
if not self.args.master or not self.args.builder: if not self.args.builder_group or not self.args.builder:
return {} return {}
path = self.PathJoin(self.chromium_src_dir, 'ios', 'build', 'bots', path = self.PathJoin(self.chromium_src_dir, 'ios', 'build', 'bots',
self.args.master, self.args.builder + '.json') self.args.builder_group, self.args.builder + '.json')
if not self.Exists(path): if not self.Exists(path):
return {} return {}
...@@ -947,7 +950,9 @@ class MetaBuildWrapper(object): ...@@ -947,7 +950,9 @@ class MetaBuildWrapper(object):
self.configs = contents['configs'] self.configs = contents['configs']
self.mixins = contents['mixins'] self.mixins = contents['mixins']
self.masters = contents.get('masters') # TODO(crbug.com/1117773): Remove 'masters' below.
self.builder_groups = (
contents.get('builder_groups') or contents.get('masters'))
self.public_artifact_builders = contents.get('public_artifact_builders') self.public_artifact_builders = contents.get('public_artifact_builders')
def ReadIsolateMap(self): def ReadIsolateMap(self):
...@@ -974,38 +979,39 @@ class MetaBuildWrapper(object): ...@@ -974,38 +979,39 @@ class MetaBuildWrapper(object):
def ConfigFromArgs(self): def ConfigFromArgs(self):
if self.args.config: if self.args.config:
if self.args.master or self.args.builder: if self.args.builder_group or self.args.builder:
raise MBErr('Can not specific both -c/--config and -m/--master or ' raise MBErr('Can not specific both -c/--config and --group '
'-b/--builder') 'or -b/--builder')
return self.args.config return self.args.config
if not self.args.master or not self.args.builder: if not self.args.builder_group or not self.args.builder:
raise MBErr('Must specify either -c/--config or ' raise MBErr('Must specify either -c/--config or '
'(-m/--master and -b/--builder)') '(--group and -b/--builder)')
if not self.args.master in self.masters: if not self.args.builder_group in self.builder_groups:
raise MBErr('Master name "%s" not found in "%s"' % raise MBErr('Builder group name "%s" not found in "%s"' %
(self.args.master, self.args.config_file)) (self.args.builder_group, self.args.config_file))
if not self.args.builder in self.masters[self.args.master]: if not self.args.builder in self.builder_groups[self.args.builder_group]:
raise MBErr('Builder name "%s" not found under masters[%s] in "%s"' % raise MBErr('Builder name "%s" not found under groups[%s] in "%s"' %
(self.args.builder, self.args.master, self.args.config_file)) (self.args.builder, self.args.builder_group,
self.args.config_file))
config = self.masters[self.args.master][self.args.builder] config = self.builder_groups[self.args.builder_group][self.args.builder]
if isinstance(config, dict): if isinstance(config, dict):
if self.args.phase is None: if self.args.phase is None:
raise MBErr('Must specify a build --phase for %s on %s' % raise MBErr('Must specify a build --phase for %s on %s' %
(self.args.builder, self.args.master)) (self.args.builder, self.args.builder_group))
phase = str(self.args.phase) phase = str(self.args.phase)
if phase not in config: if phase not in config:
raise MBErr('Phase %s doesn\'t exist for %s on %s' % raise MBErr('Phase %s doesn\'t exist for %s on %s' %
(phase, self.args.builder, self.args.master)) (phase, self.args.builder, self.args.builder_group))
return config[phase] return config[phase]
if self.args.phase is not None: if self.args.phase is not None:
raise MBErr('Must not specify a build --phase for %s on %s' % raise MBErr('Must not specify a build --phase for %s on %s' %
(self.args.builder, self.args.master)) (self.args.builder, self.args.builder_group))
return config return config
def RunGNGen(self, vals, compute_inputs_for_analyze=False, check=True): def RunGNGen(self, vals, compute_inputs_for_analyze=False, check=True):
......
...@@ -9,10 +9,10 @@ ...@@ -9,10 +9,10 @@
# * trailing commas are allowed. # * trailing commas are allowed.
{ {
# This is a map of buildbot master names -> buildbot builder names -> # This is a map of builder group names -> builder names -> config names
# config names (where each config name is a key in the 'configs' dict, # (where each config name is a key in the 'configs' dict, below). MB uses
# below). MB uses this dict to look up which config to use for a given bot. # this dict to look up which config to use for a given bot.
'masters': { 'builder_groups': {
'chrome': { 'chrome': {
'chromeos-arm-generic-beta': 'official_chromeos_arm-generic', 'chromeos-arm-generic-beta': 'official_chromeos_arm-generic',
'chromeos-arm-generic-ltc': 'official_chromeos_arm-generic', 'chromeos-arm-generic-ltc': 'official_chromeos_arm-generic',
...@@ -682,7 +682,7 @@ ...@@ -682,7 +682,7 @@
'win32-updater-builder-rel': 'updater_release_bot_x86', 'win32-updater-builder-rel': 'updater_release_bot_x86',
}, },
# TODO(crbug.com/818301): This master is going away. # TODO(crbug.com/818301): This group is going away.
'chromium.webkit': { 'chromium.webkit': {
'WebKit Linux Trusty ASAN': 'asan_lsan_release_bot', 'WebKit Linux Trusty ASAN': 'asan_lsan_release_bot',
'WebKit Linux Trusty Leak': 'release_bot', 'WebKit Linux Trusty Leak': 'release_bot',
......
...@@ -151,12 +151,12 @@ class FakeFile(object): ...@@ -151,12 +151,12 @@ class FakeFile(object):
TEST_CONFIG = """\ TEST_CONFIG = """\
{ {
'masters': { 'builder_groups': {
'chromium': {}, 'chromium': {},
'fake_master': { 'fake_builder_group': {
'fake_builder': 'rel_bot', 'fake_builder': 'rel_bot',
'fake_debug_builder': 'debug_goma', 'fake_debug_builder': 'debug_goma',
'fake_args_bot': '//build/args/bots/fake_master/fake_args_bot.gn', 'fake_args_bot': '//build/args/bots/fake_builder_group/fake_args_bot.gn',
'fake_multi_phase': { 'phase_1': 'phase_1', 'phase_2': 'phase_2'}, 'fake_multi_phase': { 'phase_1': 'phase_1', 'phase_2': 'phase_2'},
'fake_args_file': 'args_file_goma', 'fake_args_file': 'args_file_goma',
'fake_ios_error': 'ios_error', 'fake_ios_error': 'ios_error',
...@@ -205,7 +205,7 @@ TEST_BAD_CONFIG = """\ ...@@ -205,7 +205,7 @@ TEST_BAD_CONFIG = """\
'rel_bot_1': ['rel', 'chrome_with_codecs'], 'rel_bot_1': ['rel', 'chrome_with_codecs'],
'rel_bot_2': ['rel', 'bad_nested_config'], 'rel_bot_2': ['rel', 'bad_nested_config'],
}, },
'masters': { 'builder_groups': {
'chromium': { 'chromium': {
'a': 'rel_bot_1', 'a': 'rel_bot_1',
'b': 'rel_bot_2', 'b': 'rel_bot_2',
...@@ -229,9 +229,9 @@ TEST_BAD_CONFIG = """\ ...@@ -229,9 +229,9 @@ TEST_BAD_CONFIG = """\
TEST_ARGS_FILE_TWICE_CONFIG = """\ TEST_ARGS_FILE_TWICE_CONFIG = """\
{ {
'masters': { 'builder_groups': {
'chromium': {}, 'chromium': {},
'fake_master': { 'fake_builder_group': {
'fake_args_file_twice': 'args_file_twice', 'fake_args_file_twice': 'args_file_twice',
}, },
}, },
...@@ -249,9 +249,9 @@ TEST_ARGS_FILE_TWICE_CONFIG = """\ ...@@ -249,9 +249,9 @@ TEST_ARGS_FILE_TWICE_CONFIG = """\
TEST_DUP_CONFIG = """\ TEST_DUP_CONFIG = """\
{ {
'masters': { 'builder_groups': {
'chromium': {}, 'chromium': {},
'fake_master': { 'fake_builder_group': {
'fake_builder': 'some_config', 'fake_builder': 'some_config',
'other_builder': 'some_other_config', 'other_builder': 'some_other_config',
}, },
...@@ -270,7 +270,7 @@ TEST_DUP_CONFIG = """\ ...@@ -270,7 +270,7 @@ TEST_DUP_CONFIG = """\
TRYSERVER_CONFIG = """\ TRYSERVER_CONFIG = """\
{ {
'masters': { 'builder_groups': {
'not_a_tryserver': { 'not_a_tryserver': {
'fake_builder': 'fake_config', 'fake_builder': 'fake_config',
}, },
...@@ -301,7 +301,7 @@ class UnitTest(unittest.TestCase): ...@@ -301,7 +301,7 @@ class UnitTest(unittest.TestCase):
}, },
}''') }''')
mbw.files.setdefault( mbw.files.setdefault(
mbw.ToAbsPath('//build/args/bots/fake_master/fake_args_bot.gn'), mbw.ToAbsPath('//build/args/bots/fake_builder_group/fake_args_bot.gn'),
'is_debug = false\n') 'is_debug = false\n')
if files: if files:
for path, contents in files.items(): for path, contents in files.items():
...@@ -453,17 +453,17 @@ class UnitTest(unittest.TestCase): ...@@ -453,17 +453,17 @@ class UnitTest(unittest.TestCase):
'--check', mbw.out) '--check', mbw.out)
mbw = self.fake_mbw() mbw = self.fake_mbw()
self.check(['gen', '-m', 'fake_master', '-b', 'fake_args_bot', self.check(['gen', '-m', 'fake_builder_group', '-b', 'fake_args_bot',
'//out/Debug'], '//out/Debug'],
mbw=mbw, ret=0) mbw=mbw, ret=0)
# TODO(https://crbug.com/1093038): This assert is inappropriately failing. # TODO(https://crbug.com/1093038): This assert is inappropriately failing.
# self.assertEqual( # self.assertEqual(
# mbw.files['/fake_src/out/Debug/args.gn'], # mbw.files['/fake_src/out/Debug/args.gn'],
# 'import("//build/args/bots/fake_master/fake_args_bot.gn")\n') # 'import("//build/args/bots/fake_builder_group/fake_args_bot.gn")\n')
def test_gen_args_file_mixins(self): def test_gen_args_file_mixins(self):
mbw = self.fake_mbw() mbw = self.fake_mbw()
self.check(['gen', '-m', 'fake_master', '-b', 'fake_args_file', self.check(['gen', '-m', 'fake_builder_group', '-b', 'fake_args_file',
'//out/Debug'], mbw=mbw, ret=0) '//out/Debug'], mbw=mbw, ret=0)
self.assertEqual( self.assertEqual(
...@@ -474,7 +474,7 @@ class UnitTest(unittest.TestCase): ...@@ -474,7 +474,7 @@ class UnitTest(unittest.TestCase):
def test_gen_args_file_twice(self): def test_gen_args_file_twice(self):
mbw = self.fake_mbw() mbw = self.fake_mbw()
mbw.files[mbw.default_config] = TEST_ARGS_FILE_TWICE_CONFIG mbw.files[mbw.default_config] = TEST_ARGS_FILE_TWICE_CONFIG
self.check(['gen', '-m', 'fake_master', '-b', 'fake_args_file_twice', self.check(['gen', '-m', 'fake_builder_group', '-b', 'fake_args_file_twice',
'//out/Debug'], mbw=mbw, ret=1) '//out/Debug'], mbw=mbw, ret=1)
def test_gen_fails(self): def test_gen_fails(self):
...@@ -781,27 +781,27 @@ class UnitTest(unittest.TestCase): ...@@ -781,27 +781,27 @@ class UnitTest(unittest.TestCase):
def test_multiple_phases(self): def test_multiple_phases(self):
# Check that not passing a --phase to a multi-phase builder fails. # Check that not passing a --phase to a multi-phase builder fails.
mbw = self.check(['lookup', '-m', 'fake_master', '-b', 'fake_multi_phase'], mbw = self.check(['lookup', '-m', 'fake_builder_group', '-b',
ret=1) 'fake_multi_phase'], ret=1)
self.assertIn('Must specify a build --phase', mbw.out) self.assertIn('Must specify a build --phase', mbw.out)
# Check that passing a --phase to a single-phase builder fails. # Check that passing a --phase to a single-phase builder fails.
mbw = self.check(['lookup', '-m', 'fake_master', '-b', 'fake_builder', mbw = self.check(['lookup', '-m', 'fake_builder_group', '-b',
'--phase', 'phase_1'], ret=1) 'fake_builder', '--phase', 'phase_1'], ret=1)
self.assertIn('Must not specify a build --phase', mbw.out) self.assertIn('Must not specify a build --phase', mbw.out)
# Check that passing a wrong phase key to a multi-phase builder fails. # Check that passing a wrong phase key to a multi-phase builder fails.
mbw = self.check(['lookup', '-m', 'fake_master', '-b', 'fake_multi_phase', mbw = self.check(['lookup', '-m', 'fake_builder_group', '-b',
'--phase', 'wrong_phase'], ret=1) 'fake_multi_phase', '--phase', 'wrong_phase'], ret=1)
self.assertIn('Phase wrong_phase doesn\'t exist', mbw.out) self.assertIn('Phase wrong_phase doesn\'t exist', mbw.out)
# Check that passing a correct phase key to a multi-phase builder passes. # Check that passing a correct phase key to a multi-phase builder passes.
mbw = self.check(['lookup', '-m', 'fake_master', '-b', 'fake_multi_phase', mbw = self.check(['lookup', '-m', 'fake_builder_group', '-b',
'--phase', 'phase_1'], ret=0) 'fake_multi_phase', '--phase', 'phase_1'], ret=0)
self.assertIn('phase = 1', mbw.out) self.assertIn('phase = 1', mbw.out)
mbw = self.check(['lookup', '-m', 'fake_master', '-b', 'fake_multi_phase', mbw = self.check(['lookup', '-m', 'fake_builder_group', '-b',
'--phase', 'phase_2'], ret=0) 'fake_multi_phase', '--phase', 'phase_2'], ret=0)
self.assertIn('phase = 2', mbw.out) self.assertIn('phase = 2', mbw.out)
def test_recursive_lookup(self): def test_recursive_lookup(self):
...@@ -811,7 +811,7 @@ class UnitTest(unittest.TestCase): ...@@ -811,7 +811,7 @@ class UnitTest(unittest.TestCase):
'enable_antidoom_banana = true\n' 'enable_antidoom_banana = true\n'
) )
} }
self.check(['lookup', '-m', 'fake_master', '-b', 'fake_args_file', self.check(['lookup', '-m', 'fake_builder_group', '-b', 'fake_args_file',
'--recursive'], files=files, ret=0, '--recursive'], files=files, ret=0,
out=('enable_antidoom_banana = true\n' out=('enable_antidoom_banana = true\n'
'enable_doom_melon = true\n' 'enable_doom_melon = true\n'
...@@ -821,7 +821,7 @@ class UnitTest(unittest.TestCase): ...@@ -821,7 +821,7 @@ class UnitTest(unittest.TestCase):
mbw = self.fake_mbw() mbw = self.fake_mbw()
temp_dir = mbw.TempDir() temp_dir = mbw.TempDir()
self.check(['train', '--expectations-dir', temp_dir], mbw=mbw, ret=0) self.check(['train', '--expectations-dir', temp_dir], mbw=mbw, ret=0)
self.assertIn(os.path.join(temp_dir, 'fake_master.json'), mbw.files) self.assertIn(os.path.join(temp_dir, 'fake_builder_group.json'), mbw.files)
def test_validate(self): def test_validate(self):
mbw = self.fake_mbw() mbw = self.fake_mbw()
...@@ -855,7 +855,7 @@ class UnitTest(unittest.TestCase): ...@@ -855,7 +855,7 @@ class UnitTest(unittest.TestCase):
temp_dir = mbw.TempDir() temp_dir = mbw.TempDir()
self.check(['train', '--expectations-dir', temp_dir], mbw=mbw, ret=0) self.check(['train', '--expectations-dir', temp_dir], mbw=mbw, ret=0)
# Remove one of the expectation files. # Remove one of the expectation files.
mbw.files.pop(os.path.join(temp_dir, 'fake_master.json')) mbw.files.pop(os.path.join(temp_dir, 'fake_builder_group.json'))
# Now validating should fail. # Now validating should fail.
self.check(['validate', '--expectations-dir', temp_dir], mbw=mbw, ret=1) self.check(['validate', '--expectations-dir', temp_dir], mbw=mbw, ret=1)
self.assertIn('Expectations out of date', mbw.out) self.assertIn('Expectations out of date', mbw.out)
...@@ -897,11 +897,11 @@ class UnitTest(unittest.TestCase): ...@@ -897,11 +897,11 @@ class UnitTest(unittest.TestCase):
def test_ios_error_config_with_ios_json(self): def test_ios_error_config_with_ios_json(self):
"""Ensures that ios_error config finds the correct iOS JSON file for args""" """Ensures that ios_error config finds the correct iOS JSON file for args"""
files = { files = {
'/fake_src/ios/build/bots/fake_master/fake_ios_error.json': '/fake_src/ios/build/bots/fake_builder_group/fake_ios_error.json':
('{"gn_args": ["is_debug=true"]}\n') ('{"gn_args": ["is_debug=true"]}\n')
} }
mbw = self.fake_mbw(files) mbw = self.fake_mbw(files)
self.check(['lookup', '-m', 'fake_master', '-b', 'fake_ios_error'], self.check(['lookup', '-m', 'fake_builder_group', '-b', 'fake_ios_error'],
mbw=mbw, mbw=mbw,
ret=0, ret=0,
out=('\n' out=('\n'
...@@ -917,11 +917,11 @@ class UnitTest(unittest.TestCase): ...@@ -917,11 +917,11 @@ class UnitTest(unittest.TestCase):
checked. checked.
""" """
files = { files = {
'/fake_src/ios/build/bots/fake_master/fake_ios_bot.json': '/fake_src/ios/build/bots/fake_builder_group/fake_ios_bot.json':
('{"gn_args": ["is_debug=true"]}\n') ('{"gn_args": ["is_debug=true"]}\n')
} }
mbw = self.fake_mbw(files) mbw = self.fake_mbw(files)
self.check(['lookup', '-m', 'fake_master', '-b', 'fake_ios_bot'], self.check(['lookup', '-m', 'fake_builder_group', '-b', 'fake_ios_bot'],
mbw=mbw, mbw=mbw,
ret=0, ret=0,
out=('\n' out=('\n'
...@@ -937,7 +937,7 @@ class UnitTest(unittest.TestCase): ...@@ -937,7 +937,7 @@ class UnitTest(unittest.TestCase):
is ios_error, but there is no iOS JSON definition for it. is ios_error, but there is no iOS JSON definition for it.
""" """
mbw = self.fake_mbw() mbw = self.fake_mbw()
self.check(['lookup', '-m', 'fake_master', '-b', 'fake_ios_error'], self.check(['lookup', '-m', 'fake_builder_group', '-b', 'fake_ios_error'],
mbw=mbw, mbw=mbw,
ret=1) ret=1)
self.assertIn('MBErr: No iOS definition was found.', mbw.out) self.assertIn('MBErr: No iOS definition was found.', mbw.out)
...@@ -949,10 +949,10 @@ class UnitTest(unittest.TestCase): ...@@ -949,10 +949,10 @@ class UnitTest(unittest.TestCase):
doesn't exist at all. doesn't exist at all.
""" """
mbw = self.fake_mbw() mbw = self.fake_mbw()
self.check(['lookup', '-m', 'fake_master', '-b', 'random_bot'], self.check(['lookup', '-m', 'fake_builder_group', '-b', 'random_bot'],
mbw=mbw, mbw=mbw,
ret=1) ret=1)
self.assertIn('MBErr: Builder name "random_bot" not found under masters', self.assertIn('MBErr: Builder name "random_bot" not found under groups',
mbw.out) mbw.out)
......
...@@ -26,8 +26,8 @@ TEST_UNREFERENCED_MIXIN_CONFIG = """\ ...@@ -26,8 +26,8 @@ TEST_UNREFERENCED_MIXIN_CONFIG = """\
'rel_bot_1': ['rel'], 'rel_bot_1': ['rel'],
'rel_bot_2': ['rel'], 'rel_bot_2': ['rel'],
}, },
'masters': { 'builder_groups': {
'fake_master_a': { 'fake_builder_group_a': {
'fake_builder_a': 'rel_bot_1', 'fake_builder_a': 'rel_bot_1',
'fake_builder_b': 'rel_bot_2', 'fake_builder_b': 'rel_bot_2',
}, },
...@@ -50,8 +50,8 @@ TEST_UNKNOWNMIXIN_CONFIG = """\ ...@@ -50,8 +50,8 @@ TEST_UNKNOWNMIXIN_CONFIG = """\
'rel_bot_1': ['rel'], 'rel_bot_1': ['rel'],
'rel_bot_2': ['rel', 'unknown_mixin'], 'rel_bot_2': ['rel', 'unknown_mixin'],
}, },
'masters': { 'builder_groups': {
'fake_master_a': { 'fake_builder_group_a': {
'fake_builder_a': 'rel_bot_1', 'fake_builder_a': 'rel_bot_1',
'fake_builder_b': 'rel_bot_2', 'fake_builder_b': 'rel_bot_2',
}, },
...@@ -71,8 +71,8 @@ TEST_UNKNOWN_NESTED_MIXIN_CONFIG = """\ ...@@ -71,8 +71,8 @@ TEST_UNKNOWN_NESTED_MIXIN_CONFIG = """\
'rel_bot_1': ['rel', 'nested_mixin'], 'rel_bot_1': ['rel', 'nested_mixin'],
'rel_bot_2': ['rel'], 'rel_bot_2': ['rel'],
}, },
'masters': { 'builder_groups': {
'fake_master_a': { 'fake_builder_group_a': {
'fake_builder_a': 'rel_bot_1', 'fake_builder_a': 'rel_bot_1',
'fake_builder_b': 'rel_bot_2', 'fake_builder_b': 'rel_bot_2',
}, },
...@@ -96,14 +96,14 @@ TEST_UNKNOWN_NESTED_MIXIN_CONFIG = """\ ...@@ -96,14 +96,14 @@ TEST_UNKNOWN_NESTED_MIXIN_CONFIG = """\
class UnitTest(unittest.TestCase): class UnitTest(unittest.TestCase):
def test_GetAllConfigs(self): def test_GetAllConfigs(self):
configs = ast.literal_eval(mb_unittest.TEST_CONFIG) configs = ast.literal_eval(mb_unittest.TEST_CONFIG)
all_configs = validation.GetAllConfigs(configs['masters']) all_configs = validation.GetAllConfigs(configs['builder_groups'])
self.assertEqual(all_configs['rel_bot'], 'fake_master') self.assertEqual(all_configs['rel_bot'], 'fake_builder_group')
self.assertEqual(all_configs['debug_goma'], 'fake_master') self.assertEqual(all_configs['debug_goma'], 'fake_builder_group')
def test_CheckAllConfigsAndMixinsReferenced_ok(self): def test_CheckAllConfigsAndMixinsReferenced_ok(self):
configs = ast.literal_eval(mb_unittest.TEST_CONFIG) configs = ast.literal_eval(mb_unittest.TEST_CONFIG)
errs = [] errs = []
all_configs = validation.GetAllConfigs(configs['masters']) all_configs = validation.GetAllConfigs(configs['builder_groups'])
config_configs = configs['configs'] config_configs = configs['configs']
mixins = configs['mixins'] mixins = configs['mixins']
...@@ -115,7 +115,7 @@ class UnitTest(unittest.TestCase): ...@@ -115,7 +115,7 @@ class UnitTest(unittest.TestCase):
def test_CheckAllConfigsAndMixinsReferenced_unreferenced(self): def test_CheckAllConfigsAndMixinsReferenced_unreferenced(self):
configs = ast.literal_eval(TEST_UNREFERENCED_MIXIN_CONFIG) configs = ast.literal_eval(TEST_UNREFERENCED_MIXIN_CONFIG)
errs = [] errs = []
all_configs = validation.GetAllConfigs(configs['masters']) all_configs = validation.GetAllConfigs(configs['builder_groups'])
config_configs = configs['configs'] config_configs = configs['configs']
mixins = configs['mixins'] mixins = configs['mixins']
...@@ -127,7 +127,7 @@ class UnitTest(unittest.TestCase): ...@@ -127,7 +127,7 @@ class UnitTest(unittest.TestCase):
def test_CheckAllConfigsAndMixinsReferenced_unknown(self): def test_CheckAllConfigsAndMixinsReferenced_unknown(self):
configs = ast.literal_eval(TEST_UNKNOWNMIXIN_CONFIG) configs = ast.literal_eval(TEST_UNKNOWNMIXIN_CONFIG)
errs = [] errs = []
all_configs = validation.GetAllConfigs(configs['masters']) all_configs = validation.GetAllConfigs(configs['builder_groups'])
config_configs = configs['configs'] config_configs = configs['configs']
mixins = configs['mixins'] mixins = configs['mixins']
...@@ -140,7 +140,7 @@ class UnitTest(unittest.TestCase): ...@@ -140,7 +140,7 @@ class UnitTest(unittest.TestCase):
def test_CheckAllConfigsAndMixinsReferenced_unknown_nested(self): def test_CheckAllConfigsAndMixinsReferenced_unknown_nested(self):
configs = ast.literal_eval(TEST_UNKNOWN_NESTED_MIXIN_CONFIG) configs = ast.literal_eval(TEST_UNKNOWN_NESTED_MIXIN_CONFIG)
errs = [] errs = []
all_configs = validation.GetAllConfigs(configs['masters']) all_configs = validation.GetAllConfigs(configs['builder_groups'])
config_configs = configs['configs'] config_configs = configs['configs']
mixins = configs['mixins'] mixins = configs['mixins']
...@@ -154,7 +154,7 @@ class UnitTest(unittest.TestCase): ...@@ -154,7 +154,7 @@ class UnitTest(unittest.TestCase):
def test_CheckAllConfigsAndMixinsReferenced_unused(self): def test_CheckAllConfigsAndMixinsReferenced_unused(self):
configs = ast.literal_eval(TEST_UNKNOWN_NESTED_MIXIN_CONFIG) configs = ast.literal_eval(TEST_UNKNOWN_NESTED_MIXIN_CONFIG)
errs = [] errs = []
all_configs = validation.GetAllConfigs(configs['masters']) all_configs = validation.GetAllConfigs(configs['builder_groups'])
config_configs = configs['configs'] config_configs = configs['configs']
mixins = configs['mixins'] mixins = configs['mixins']
...@@ -168,11 +168,12 @@ class UnitTest(unittest.TestCase): ...@@ -168,11 +168,12 @@ class UnitTest(unittest.TestCase):
def test_EnsureNoProprietaryMixins(self): def test_EnsureNoProprietaryMixins(self):
bad_configs = ast.literal_eval(mb_unittest.TEST_BAD_CONFIG) bad_configs = ast.literal_eval(mb_unittest.TEST_BAD_CONFIG)
errs = [] errs = []
masters = bad_configs['masters'] builder_groups = bad_configs['builder_groups']
mixins = bad_configs['mixins'] mixins = bad_configs['mixins']
config_configs = bad_configs['configs'] config_configs = bad_configs['configs']
validation.EnsureNoProprietaryMixins(errs, masters, config_configs, mixins) validation.EnsureNoProprietaryMixins(errs, builder_groups, config_configs,
mixins)
self.assertIn( self.assertIn(
'Public artifact builder "a" ' 'Public artifact builder "a" '
...@@ -186,7 +187,7 @@ class UnitTest(unittest.TestCase): ...@@ -186,7 +187,7 @@ class UnitTest(unittest.TestCase):
configs = ast.literal_eval(mb_unittest.TEST_CONFIG) configs = ast.literal_eval(mb_unittest.TEST_CONFIG)
config_configs = configs['configs'] config_configs = configs['configs']
mixins = configs['mixins'] mixins = configs['mixins']
grouping = configs['masters'] grouping = configs['builder_groups']
errs = [] errs = []
validation.CheckDuplicateConfigs(errs, config_configs, mixins, grouping, validation.CheckDuplicateConfigs(errs, config_configs, mixins, grouping,
...@@ -198,7 +199,7 @@ class UnitTest(unittest.TestCase): ...@@ -198,7 +199,7 @@ class UnitTest(unittest.TestCase):
configs = ast.literal_eval(mb_unittest.TEST_DUP_CONFIG) configs = ast.literal_eval(mb_unittest.TEST_DUP_CONFIG)
config_configs = configs['configs'] config_configs = configs['configs']
mixins = configs['mixins'] mixins = configs['mixins']
grouping = configs['masters'] grouping = configs['builder_groups']
errs = [] errs = []
validation.CheckDuplicateConfigs(errs, config_configs, mixins, grouping, validation.CheckDuplicateConfigs(errs, config_configs, mixins, grouping,
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment