Commit 71a945b8 authored by Egor Pasko's avatar Egor Pasko Committed by Commit Bot

benchmarks/startup_mobile: cool down between runs

I found that cooling down between runs helps a lot with the noise.
Applying the usual threshold of 35Celsius, identical to the one used by
story_runner.

My testing methodology is simple: connect a N5X and run this with bash:

COMMON_LOG="logs/noise"
for ((i=0;i<10;i++)); do
  OUTPUT="$COMMON_LOG/old$i"
  mkdir -p "$OUTPUT"
  CHROMIUM_OUTPUT_DIR=gn_android/ReleaseOfficial \
      tools/perf/run_benchmark -v \
      experimental.startup.android.coldish \
      --browser=android-chrome --output-dir="$OUTPUT" 2>&1 \
      | tee "$OUTPUT/run.log"

  OUTPUT="$COMMON_LOG/new$i"
  mkdir -p "$OUTPUT"
  CHROMIUM_OUTPUT_DIR=gn_android/ReleaseOfficial \
      tools/perf/run_benchmark -v \
      experimental.startup.mobile \
      --browser=android-chrome --output-dir="$OUTPUT" 2>&1 \
      | tee "$OUTPUT/run.log"
done

On my N5X we appear to have to wait ~2 times per 10 Chrome starts to
cool down the device, which is a lot of waiting. My desk is not the
coldest across offices though, close to 25C right now.

Noise is still an issue for 2-3 last measurements out of every batch of
10. These last runs are typically 200-300ms slower than the first 5+
runs (for messageloop start). I suspect there are still effects of
overheating.  The battery cutoff at 35C is probably not providing
sufficient time to cool the CPU down in the new benchmark.

But why is it 'good enough' for the old benchmark? I think the old way
performs more dummy intermediate steps (reinitializing commandline/trace
file, restarting forwarder/WPR, etc.) to allow the device to cool down.

Frankly, upwards trend is present on the old benchmark as well, but more
around +50ms over 10 runs. While it is not too bad at the current stage,
it may become unpleasant at bisect with more iterations, or in a room
that is slightly hotter. So I'll probably take a look at limiting
the clock frequency for big cores, like we do for N5.

Bug: 760498
Change-Id: Id050cf54c6ea21c51acdfbb1c06f20023d909454
Reviewed-on: https://chromium-review.googlesource.com/1180202Reviewed-by: default avatarJuan Antonio Navarro Pérez <perezju@chromium.org>
Reviewed-by: default avatarNed Nguyen <nednguyen@google.com>
Commit-Queue: Egor Pasko <pasko@chromium.org>
Cr-Commit-Position: refs/heads/master@{#584742}
parent a718f913
......@@ -84,6 +84,7 @@ class _MobileStartupSharedState(story_module.SharedState):
# TODO(crbug.com/811244): Determine whether this ensures the page cache is
# cleared after |FlushOsPageCaches()| returns.
self._possible_browser.FlushOsPageCaches()
self.platform.WaitForBatteryTemperature(35)
self.platform.StartActivity(
intent.Intent(package=self._possible_browser.settings.package,
activity=self._possible_browser.settings.activity,
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment