Commit 8294a2f3 authored by Daniel Cheng's avatar Daniel Cheng Committed by Commit Bot

[web tests] Concatenate regex to speed up web test result JSON merging.

The JSON files that are merged by the helper script generally contain a
lot of values. The regex matcher in Python is quite optimized, so it's
faster to combine identical cases with '|' rather than adding multiple
merge helpers that execute the same thing.

Savings when run locally on my dev machine are around 500ms-1000ms.

Bug: 1014845
Change-Id: I36af02140b5efa2cbd306f7f1fb6c26f93f37a29
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1894416Reviewed-by: default avatarKent Tamura <tkent@chromium.org>
Commit-Queue: Kent Tamura <tkent@chromium.org>
Auto-Submit: Daniel Cheng <dcheng@chromium.org>
Cr-Commit-Position: refs/heads/master@{#711646}
parent fb80839d
......@@ -551,10 +551,13 @@ class JSONTestResultsMerger(JSONMerger):
':random_order_seed$',
':version$',
]
for match_name in matching:
self.add_helper(
NameRegexMatch(match_name),
self.merge_equal)
# Note: the regex matcher is quite fast, so take advantage of it to
# combine identical actions into one. The JSON files contain many keys,
# and the cost of iterating over and executing multiple identical
# helpers is measurable.
self.add_helper(
NameRegexMatch('|'.join(matching)),
self.merge_equal)
# These keys are accumulated sums we want to add together.
addable = [
......@@ -567,10 +570,9 @@ class JSONTestResultsMerger(JSONMerger):
# All keys inside the num_failures_by_type entry.
':num_failures_by_type:',
]
for match_name in addable:
self.add_helper(
NameRegexMatch(match_name),
lambda o, name=None: sum(o))
self.add_helper(
NameRegexMatch('|'.join(addable)),
lambda o, name=None: sum(o))
# If any shard is interrupted, mark the whole thing as interrupted.
self.add_helper(
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment