Commit 8294a2f3 authored by Daniel Cheng's avatar Daniel Cheng Committed by Commit Bot

[web tests] Concatenate regex to speed up web test result JSON merging.

The JSON files that are merged by the helper script generally contain a
lot of values. The regex matcher in Python is quite optimized, so it's
faster to combine identical cases with '|' rather than adding multiple
merge helpers that execute the same thing.

Savings when run locally on my dev machine are around 500ms-1000ms.

Bug: 1014845
Change-Id: I36af02140b5efa2cbd306f7f1fb6c26f93f37a29
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1894416Reviewed-by: default avatarKent Tamura <tkent@chromium.org>
Commit-Queue: Kent Tamura <tkent@chromium.org>
Auto-Submit: Daniel Cheng <dcheng@chromium.org>
Cr-Commit-Position: refs/heads/master@{#711646}
parent fb80839d
...@@ -551,10 +551,13 @@ class JSONTestResultsMerger(JSONMerger): ...@@ -551,10 +551,13 @@ class JSONTestResultsMerger(JSONMerger):
':random_order_seed$', ':random_order_seed$',
':version$', ':version$',
] ]
for match_name in matching: # Note: the regex matcher is quite fast, so take advantage of it to
self.add_helper( # combine identical actions into one. The JSON files contain many keys,
NameRegexMatch(match_name), # and the cost of iterating over and executing multiple identical
self.merge_equal) # helpers is measurable.
self.add_helper(
NameRegexMatch('|'.join(matching)),
self.merge_equal)
# These keys are accumulated sums we want to add together. # These keys are accumulated sums we want to add together.
addable = [ addable = [
...@@ -567,10 +570,9 @@ class JSONTestResultsMerger(JSONMerger): ...@@ -567,10 +570,9 @@ class JSONTestResultsMerger(JSONMerger):
# All keys inside the num_failures_by_type entry. # All keys inside the num_failures_by_type entry.
':num_failures_by_type:', ':num_failures_by_type:',
] ]
for match_name in addable: self.add_helper(
self.add_helper( NameRegexMatch('|'.join(addable)),
NameRegexMatch(match_name), lambda o, name=None: sum(o))
lambda o, name=None: sum(o))
# If any shard is interrupted, mark the whole thing as interrupted. # If any shard is interrupted, mark the whole thing as interrupted.
self.add_helper( self.add_helper(
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment