Commit a92b95bb authored by Ned Nguyen's avatar Ned Nguyen Committed by Commit Bot

Maximize the number of parallel processes for processing & uploading perf data

A big portion of perf data uploading latency is due to I/O operation (uploading),
hence maximize the number of parallel processes to improve the wall-time.

Bug:857283
Change-Id: I0c3246849676a56801fe99846ade7265d13d94c0

TBR=simonhatch@chromium.org

Change-Id: I0c3246849676a56801fe99846ade7265d13d94c0

NOTRY=true  # android_arm64_dbg_recipe flake

Change-Id: I0c3246849676a56801fe99846ade7265d13d94c0
Reviewed-on: https://chromium-review.googlesource.com/1170083
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: default avatarNed Nguyen <nednguyen@google.com>
Reviewed-by: default avatarEthan Kuefner <eakuefner@chromium.org>
Cr-Commit-Position: refs/heads/master@{#581991}
parent b3fa0931
......@@ -404,8 +404,10 @@ def _handle_perf_results(
build_properties, output_json_file, service_account_file))
# Kick off the uploads in mutliple processes
cpus = mp.cpu_count()
pool = mp.Pool(cpus)
# Maximimze the parallelization by setting the number of parallel processes
# to the number of invocations. This is because a significant part of
# uploading are I/Os bounds.
pool = mp.Pool(len(invocations))
try:
async_result = pool.map_async(
_upload_individual_benchmark, invocations)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment