Ignore untriaged images from tests with grace periods

Adds an '"ignore": "1"' pair to the JSON for GPU pixel tests if a test
has a grace period active. This is so that newly added noisy tests do
not affect unrelated CLs by causing Gold to comment on them because they
happened to produce a new image due to bad luck.

Bug: 1081942
Change-Id: Ife8dbe1baa349c203a4e95d7a152cb42d29d384f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2199908Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#768520}

content/test/gpu/gpu_tests/skia_gold_integration_test_base.py

@@ -368,6 +368,12 @@ class SkiaGoldIntegrationTestBase(gpu_integration_test.GpuIntegrationTest):
         'msaa': str(img_params.msaa),
         'model_name': _ToNonEmptyStrOrNone(img_params.model_name),
     }
+    # If we have a grace period active, then the test is potentially flaky.
+    # Include a pair that will cause Gold to ignore any untriaged images, which
+    # will prevent it from automatically commenting on unrelated CLs that happen
+    # to produce a new image.
+    if _GracePeriodActive(page):
+      gpu_keys['ignore'] = '1'
     return gpu_keys
 
 # TODO(crbug.com/1076144): This is due for a refactor, likely similar to how
@@ -530,7 +536,7 @@ class SkiaGoldIntegrationTestBase(gpu_integration_test.GpuIntegrationTest):
       return False
     # Don't surface if the test was recently added and we're still within its
     # grace period.
-    if page.grace_period_end and date.today() <= page.grace_period_end:
+    if _GracePeriodActive(page):
       return False
     return True
 
@@ -636,6 +642,19 @@ def _ToNonEmptyStrOrNone(val):
   return 'None' if val == '' else str(val)
 
 
+def _GracePeriodActive(page):
+  """Returns whether a grace period is currently active for a test.
+
+  Args:
+    page: The GPU PixelTestPage object for the test in question.
+
+  Returns:
+    True if a grace period is defined for |page| and has not yet expired.
+    Otherwise, False.
+  """
+  return page.grace_period_end and date.today() <= page.grace_period_end
+
+
 def load_tests(loader, tests, pattern):
   del loader, tests, pattern  # Unused.
   return gpu_integration_test.LoadAllTestsInModule(sys.modules[__name__])

docs/gpu/gpu_testing.md

@@ -562,6 +562,13 @@ into:
 
 `https://chrome-gpu-gold.skia.org/search?query=name%3D<test name>`
 
+**NOTE** If you have a grace period active for your test, then Gold will be told
+to ignore results for the test. This is so that it does not comment on unrelated
+CLs about untriaged images if your test is noisy. Images will still be uploaded
+to Gold and can be triaged, but will not show up on the main page's untriaged
+image list, and you will need to enable the "Ignored" toggle at the top of the
+page when looking at the triage page specific to your test.
+
 ## Stamping out Flakiness
 
 It's critically important to aggressively investigate and eliminate the root