Temporarily increase retry limit in test_launcher

In bot mode, test_launcher would normally retry tests up to 3 times.
This is sufficient to keep large test suites like browser_tests under
1% flakiness rate if the flakiness of individual tests is also within
1%. However, some tests especially in browser_tests are up to 33%
flaky, making the browser_tests suite 40% flaky.

Subsequent top-level retries effectively double the runtime of all
bots running these tests, and we need a solution to shorten the try
job runtime.

A longer-term solution is the ignorer bot (see the bug), but until
it's deployed, a quick hack is to increase the number of retries so
statistically we should see about 1% flakiness of browser_tests at the
expense of a slight runtime increase.

R=sergiyb@chromium.org
BUG=390600, 395189

Review URL: https://codereview.chromium.org/393283006

git-svn-id: svn://svn.chromium.org/chrome/trunk/src@284650 0039d316-1c4b-4281-b951-d872f2087c98
parent 5748b780
......@@ -343,7 +343,9 @@ TestLauncher::TestLauncher(TestLauncherDelegate* launcher_delegate,
// Enable test retries by default for bots. This can be still overridden
// from command line using --test-launcher-retry-limit flag.
retry_limit_ = 3;
// TODO(sergeyberezin): decrease retry_limit_ back to 3 when the
// ignorer_bot is deployed to automatically disable flaky tests.
retry_limit_ = 12;
} else {
// Default to serial test execution if not running on a bot. This makes it
// possible to disable stdio redirection and can still be overridden with
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment