Commit e3c8d4c2 authored by Caleb Rouleau's avatar Caleb Rouleau Committed by Commit Bot

[Benchmarking] Abridge nexus5x system_health.memory in place.

Note that this will cause some of the stories of the
system_health.memory_mobile benchmark to stop being run on
nexus5x bot. It will also move some stories from one
physical machine to another of the same model. Since the
silicon lottery principle means that devices of the same
model sometimes have different performance characteristics
this may cause false positive improvements or regressions
do not revert this change if those regressions happen.

Note even though bot_platforms.py can do automatic
resharding, I decided to do the resharding manually
to avoid shuffling unrelated benchmarks and because it seems
that the sharding system is for some reason expecting
system_health.memory_mobile to run faster than it already
does (which is likely because that benchmark has a rather long
metrics calculation phase, which I suspect is not covered
in the timing data.)
I ran
./run_benchmark list system_health.memory_mobile --browser=android --json=~/listed.json
to figure out that there are 28 stories in system_health.memory_mobile with
the health check tag, so that's how I know how to manually reshard.

Bug: 1030788
Change-Id: I0735adc781c860a1dbbbbc3608e740e34f5f48ea
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1954354Reviewed-by: default avatarJuan Antonio Navarro Pérez <perezju@chromium.org>
Commit-Queue: Caleb Rouleau <crouleau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#722485}
parent 10620d4b
......@@ -184,10 +184,14 @@ _ANDROID_GO_BENCHMARK_CONFIGS = frozenset([
_ANDROID_GO_WEBVIEW_BENCHMARK_CONFIGS = _ANDROID_GO_BENCHMARK_CONFIGS
_ANDROID_NEXUS_5_BENCHMARK_CONFIGS = _OFFICIAL_EXCEPT_DISPLAY_LOCKING_JETSTREAM2
_ANDROID_NEXUS_5X_BENCHMARK_CONFIGS = (
_OFFICIAL_EXCEPT_JETSTREAM2 -
_OFFICIAL_EXCEPT_JETSTREAM2
# Remove unabridged rendering benchmark and replace with abridged benchmark.
frozenset([_GetBenchmarkConfig('rendering.mobile')]) |
frozenset([_GetBenchmarkConfig('rendering.mobile', True)]))
- frozenset([_GetBenchmarkConfig('rendering.mobile')])
| frozenset([_GetBenchmarkConfig('rendering.mobile', True)])
# Remove unabridged system health memory benchmark and replace with abridged
# benchmark: crbug.com/1030788
- frozenset([_GetBenchmarkConfig('system_health.memory_mobile')])
| frozenset([_GetBenchmarkConfig('system_health.memory_mobile', True)]))
_ANDROID_NEXUS_5X_WEBVIEW_BENCHMARK_CONFIGS = (
_OFFICIAL_EXCEPT_DISPLAY_LOCKING_JETSTREAM2)
_ANDROID_NEXUS_6_WEBVIEW_BENCHMARK_CONFIGS = (
......
......@@ -130,34 +130,34 @@
"abridged": false
},
"system_health.memory_mobile": {
"end": 14,
"abridged": false
"end": 8,
"abridged": true
}
}
},
"6": {
"benchmarks": {
"system_health.memory_mobile": {
"begin": 14,
"end": 36,
"abridged": false
"begin": 8,
"end": 16,
"abridged": true
}
}
},
"7": {
"benchmarks": {
"system_health.memory_mobile": {
"begin": 36,
"end": 78,
"abridged": false
"begin": 16,
"end": 24,
"abridged": true
}
}
},
"8": {
"benchmarks": {
"system_health.memory_mobile": {
"begin": 78,
"abridged": false
"begin": 24,
"abridged": true
},
"system_health.webview_startup": {
"abridged": false
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment