Mod a hash of test names to determine a shard, rather than the test index.
This keeps tests on a particular shard, until their name changes. Currently tests can move to a different waterfall bot between runs; whenever a test is added or removed "before" it in the test suite. This makes it very hard to track failure history of a particular test, since it can "disappear" from prior builds on that bot. Also, since the shards do not have synced cycles, a revision that breaks test "A" might never be in a blamelist. E.g. Bad revision X breaks test "A", good revision X+1 adds unrelated test "B". Blamelists bot0:[X],[X+1] and bot1:[X, X+1]. Only bot0 will run test "A", and only on its second run, but it will not blame revision "X". Note "X" could even be the revision that adds test "A". Hashing the name does not guarantee an even distribution of tests. However, the duration of a test is not predictable, so the change to load variance should balance out except in pathological cases or very small test suites. BUG=372461 TBR=phajdan.jr@chromium.org Review URL: https://codereview.chromium.org/292973006 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@272920 0039d316-1c4b-4281-b951-d872f2087c98
Showing
Please register or sign in to comment