• tapted@chromium.org's avatar
    Mod a hash of test names to determine a shard, rather than the test index. · d04140a5
    tapted@chromium.org authored
    This keeps tests on a particular shard, until their name changes.
    
    Currently tests can move to a different waterfall bot between runs;
    whenever a test is added or removed "before" it in the test suite. This
    makes it very hard to track failure history of a particular test, since
    it can "disappear" from prior builds on that bot.
    
    Also, since the shards do not have synced cycles, a revision that breaks
    test "A" might never be in a blamelist. E.g. Bad revision X breaks test
    "A", good revision X+1 adds unrelated test "B". Blamelists
    bot0:[X],[X+1] and bot1:[X, X+1]. Only bot0 will run test "A", and only
    on its second run, but it will not blame revision "X". Note "X" could
    even be the revision that adds test "A".
    
    Hashing the name does not guarantee an even distribution of tests.
    However, the duration of a test is not predictable, so the change to
    load variance should balance out except in pathological cases or very
    small test suites.
    
    BUG=372461
    TBR=phajdan.jr@chromium.org
    
    Review URL: https://codereview.chromium.org/292973006
    
    git-svn-id: svn://svn.chromium.org/chrome/trunk/src@272920 0039d316-1c4b-4281-b951-d872f2087c98
    d04140a5
test_launcher.cc 34.3 KB