Perf infra documentation updates

- Add overview of how perf builders, testers, and processors work
- Add tip on how to debug tests

Bug: chromium:1143048
Change-Id: Iebb8c72b47a5ef1dbc5c9171e3fa7148ebb202b8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2503652
Commit-Queue: Ryan Heise <heiserya@google.com>
Auto-Submit: Ryan Heise <heiserya@google.com>
Reviewed-by: John Chen <johnchen@chromium.org>
Cr-Commit-Position: refs/heads/master@{#821765}

docs/speed/perf_waterfall.md

@@ -14,6 +14,32 @@ of speed:
 > "If you make a change that regresses measured performance, you will be
 > required to fix it or revert".
 
+## How It Works
+
+The perf waterfall is split into three stages - builders, testers, and processors.
+
+### Builders
+
+For each commit to the Chromium repo, a Builder is invoked. The builder builds
+Google Chrome, acquires test assets, and bundles everything up to be passed
+along to Testers. We execute a builder for each platform we support. To ensure
+we can keep up with the rapid flow of commits, we have a set of builders per
+platform, each building a different commit.
+
+### Testers
+
+For each platform, we have a single tester job running continuously. Each run of
+a tester covers the set of commits from the end commit of the previous
+invocation to the latest _built_ commit. Each tester has one or more subdevices
+(shards). The tester delegates a subset of tests to each shard and aggregates
+high-level results (pass/fail, runtimes).
+
+### Processors
+
+Processors analyze the raw data generated by each Tester and convert it into a form that can be utilized
+by the [Chrome Performance Dashboard](https://chromeperf.appspot.com/report).
+For some jobs, this work is executed by the tester instead.
+
 ## Contact
 
   * You can reach the Chromium performance sheriffs at perf-sheriffs@chromium.org.

tools/perf/README.md

@@ -39,6 +39,11 @@ provides the ability to remotely run A/B benchmarks using any platform available
 in our lab. It will run a benchmark for as many iterations as needed to get a
 statistically significant result, then visualize it.
 
+If your're trying to debug a test or figure out how the infrastructure works,
+the easiest way is to set up the debugger in VSCode (guide
+[here](../../docs/vscode_python.md))] and set a breakpoint in
+/tools/perf/core/benchmark_runner.py.
+
 ## Creating New Tests (stories)
 
 [This document](https://chromium.googlesource.com/catapult.git/+/HEAD/telemetry)