Commit ba2e4aa6 authored by Sergey Ulanov's avatar Sergey Ulanov Committed by Commit Bot

Increase Chi-square limit in ASR tests

Previously ASR tests were asserting that Chi-square value for each bit
were below 24.0. On 64-bit platforms that condition is tested about
19000 times, so probability that at least one test fails was about 2%.
Increased the limit to 35.

Bug: 811881
Change-Id: Ia192ccf62abdd125053a0d005e632d55e8374538
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1642109
Commit-Queue: Sergey Ulanov <sergeyu@chromium.org>
Reviewed-by: default avatarWez <wez@chromium.org>
Reviewed-by: default avatarBill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/heads/master@{#665991}
parent 3c173072
......@@ -180,9 +180,10 @@ void RandomBitCorrelation(int random_bit) {
// Chi squared analysis for k = 2 (2, states: same/not-same) and one
// degree of freedom (k - 1).
double chi_squared = ChiSquared(m, kRepeats);
// For 1 degree of freedom this corresponds to 1 in a million. We are
// running ~8000 tests, so that would be surprising.
CHECK_GE(24, chi_squared);
// For k=2 probability of Chi^2 < 35 is p=3.338e-9. This condition is
// tested ~19000 times, so probability of it failing randomly per one
// base_unittests run is (1 - (1 - p) ^ 19000) ~= 6e-5.
CHECK_LE(chi_squared, 35.0);
// If the predictor bit is a fixed 0 or 1 then it makes no sense to
// repeat the test with a different age.
if (predictor_bit < 0)
......@@ -191,9 +192,6 @@ void RandomBitCorrelation(int random_bit) {
}
}
// TODO(crbug.com/811881): These are flaky on Fuchsia
#if !defined(OS_FUCHSIA)
// Tests are fairly slow, so give each random bit its own test.
#define TEST_RANDOM_BIT(BIT) \
TEST(AddressSpaceRandomizationTest, RandomBitCorrelations##BIT) { \
......@@ -242,8 +240,6 @@ TEST_RANDOM_BIT(48)
// No platforms have more than 48 address bits.
#endif // defined(ARCH_CPU_64_BITS)
#endif // defined(OS_FUCHSIA)
#undef TEST_RANDOM_BIT
} // namespace base
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment