Commit fdf80182 authored by Alexei Svitkine's avatar Alexei Svitkine Committed by Commit Bot

Improve histograms presubmit perf by memoizing dirmd lookups.

This addresses a performance regression where histograms
presubmits were taking > 2 minutes after:
https://chromium-review.googlesource.com/c/chromium/src/+/2450734

This CL adds memoization to dirmd invocations because only a few
distinct OWNERS files are specified as owners of histograms, so
we can reduce the amount of subprocess invocations significantly.

On my machine, ./tools/metrics/histograms/validate_format.py
goes from 2m8s to 25s with this change with Python 2.

Bug: 1142714
Change-Id: I1f60c60b7e6014008244c214ec7302f40e58517a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2505593Reviewed-by: default avatarCaitlin Fischer <caitlinfischer@google.com>
Commit-Queue: Alexei Svitkine <asvitkine@chromium.org>
Cr-Commit-Position: refs/heads/master@{#821756}
parent f99df198
...@@ -212,6 +212,20 @@ def _ComponentFromDirmd(json_data, subpath): ...@@ -212,6 +212,20 @@ def _ComponentFromDirmd(json_data, subpath):
{}).get('component', '') {}).get('component', '')
# Memoize decorator from: https://stackoverflow.com/a/1988826
# TODO(asvitkine): Replace with @functools.cache once we're on Python 3.9+.
class Memoize:
def __init__(self, f):
self.f = f
self.memo = {}
def __call__(self, *args):
if not args in self.memo:
self.memo[args] = self.f(*args)
return self.memo[args]
@Memoize
def _ExtractComponentViaDirmd(path): def _ExtractComponentViaDirmd(path):
"""Returns the component for monorail issues at the given path. """Returns the component for monorail issues at the given path.
...@@ -223,7 +237,7 @@ def _ExtractComponentViaDirmd(path): ...@@ -223,7 +237,7 @@ def _ExtractComponentViaDirmd(path):
Returns an empty string if no component can be extracted. Returns an empty string if no component can be extracted.
Args: Args:
path: The path to an directory to query, e.g. 'src/storage'. path: The path to a directory to query, e.g. 'src/storage'.
""" """
# Verify that the paths are absolute and the root is a parent of the # Verify that the paths are absolute and the root is a parent of the
# passed in path. # passed in path.
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment