Commit ba2e4aa6 authored by Sergey Ulanov's avatar Sergey Ulanov Committed by Commit Bot

Increase Chi-square limit in ASR tests

Previously ASR tests were asserting that Chi-square value for each bit
were below 24.0. On 64-bit platforms that condition is tested about
19000 times, so probability that at least one test fails was about 2%.
Increased the limit to 35.

Bug: 811881
Change-Id: Ia192ccf62abdd125053a0d005e632d55e8374538
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1642109
Commit-Queue: Sergey Ulanov <sergeyu@chromium.org>
Reviewed-by: default avatarWez <wez@chromium.org>
Reviewed-by: default avatarBill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/heads/master@{#665991}
parent 3c173072
...@@ -180,9 +180,10 @@ void RandomBitCorrelation(int random_bit) { ...@@ -180,9 +180,10 @@ void RandomBitCorrelation(int random_bit) {
// Chi squared analysis for k = 2 (2, states: same/not-same) and one // Chi squared analysis for k = 2 (2, states: same/not-same) and one
// degree of freedom (k - 1). // degree of freedom (k - 1).
double chi_squared = ChiSquared(m, kRepeats); double chi_squared = ChiSquared(m, kRepeats);
// For 1 degree of freedom this corresponds to 1 in a million. We are // For k=2 probability of Chi^2 < 35 is p=3.338e-9. This condition is
// running ~8000 tests, so that would be surprising. // tested ~19000 times, so probability of it failing randomly per one
CHECK_GE(24, chi_squared); // base_unittests run is (1 - (1 - p) ^ 19000) ~= 6e-5.
CHECK_LE(chi_squared, 35.0);
// If the predictor bit is a fixed 0 or 1 then it makes no sense to // If the predictor bit is a fixed 0 or 1 then it makes no sense to
// repeat the test with a different age. // repeat the test with a different age.
if (predictor_bit < 0) if (predictor_bit < 0)
...@@ -191,9 +192,6 @@ void RandomBitCorrelation(int random_bit) { ...@@ -191,9 +192,6 @@ void RandomBitCorrelation(int random_bit) {
} }
} }
// TODO(crbug.com/811881): These are flaky on Fuchsia
#if !defined(OS_FUCHSIA)
// Tests are fairly slow, so give each random bit its own test. // Tests are fairly slow, so give each random bit its own test.
#define TEST_RANDOM_BIT(BIT) \ #define TEST_RANDOM_BIT(BIT) \
TEST(AddressSpaceRandomizationTest, RandomBitCorrelations##BIT) { \ TEST(AddressSpaceRandomizationTest, RandomBitCorrelations##BIT) { \
...@@ -242,8 +240,6 @@ TEST_RANDOM_BIT(48) ...@@ -242,8 +240,6 @@ TEST_RANDOM_BIT(48)
// No platforms have more than 48 address bits. // No platforms have more than 48 address bits.
#endif // defined(ARCH_CPU_64_BITS) #endif // defined(ARCH_CPU_64_BITS)
#endif // defined(OS_FUCHSIA)
#undef TEST_RANDOM_BIT #undef TEST_RANDOM_BIT
} // namespace base } // namespace base
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment