[libFuzzer] Improve docs

Improve efficient_fuzzer.md by making the following changes:
1. Get rid of static corpus instructions (just use regular corpus)
2. Add details about authenticating to use ClusterFuzz buckets.
3. Eliminate extra newlines
4. Be consistent about the example fuzzer name
5. Mention dict should be ascii encoded

Improve clusterfuzz.md by linking directly to coverage page and by
linking to most recent stats.

Change-Id: I9990ab8ab4811445214735e65a6e8d2bd2319179
Reviewed-on: https://chromium-review.googlesource.com/c/1323272
Commit-Queue: Jonathan Metzman <metzman@chromium.org>
Reviewed-by: Max Moroz <mmoroz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#608455}

testing/libfuzzer/clusterfuzz.md

@@ -8,8 +8,8 @@ Googlers can read more [here](https://goto.google.com/clusterfuzz).
 ## Status Links
 
 * [Buildbot] - status of all libFuzzer builds.
-* [ClusterFuzz Fuzzer Status] - fuzzing metrics, links to crashes and coverage
-reports.
+* [ClusterFuzz Fuzzer Status] - fuzzing metrics.
+* [Code Coverage] - code coverage of libFuzzer targets in Chrome.
 * [ClusterFuzz libFuzzer Logs] - individual fuzz target run logs.
 * [Corpus GCS Bucket] - current corpus for each fuzz target. Can be used to
 upload bootstrapped corpus.
@@ -46,8 +46,9 @@ goes into general corpus directory. Seed corpus is read-only.
 
 
 [Buildbot]: https://build.chromium.org/p/chromium.fyi/buildslaves/slave43-c1
+[Code Coverage]: https://chromium-coverage.appspot.com/reports/latest_fuzzers_only/linux/index.html
 [chromium_libfuzzer.py]: https://code.google.com/p/chromium/codesearch#chromium/build/scripts/slave/recipes/chromium_libfuzzer.py
-[ClusterFuzz Fuzzer Status]: https://clusterfuzz.com/fuzzer-stats
+[ClusterFuzz Fuzzer Stats]: https://clusterfuzz.com/v2/fuzzer-stats/by-fuzzer/fuzzer/libFuzzer/job/libfuzzer_chrome_asan
 [ClusterFuzz libFuzzer Logs]: https://console.cloud.google.com/storage/browser/clusterfuzz-libfuzzer-logs
 [Corpus GCS Bucket]: https://console.cloud.google.com/storage/clusterfuzz-corpus/libfuzzer
 [fuzzer_test.gni]: https://code.google.com/p/chromium/codesearch#chromium/src/testing/libfuzzer/fuzzer_test.gni

testing/libfuzzer/efficient_fuzzer.md

@@ -34,7 +34,6 @@ There are other ways that are useful in some cases, but not always applicable:
 * [Custom Options](#Custom-Options)
 * [Custom Build](#Custom-Build)
 
-
 ## Execution Speed
 
 Fuzz target speed is calculated in executions per second. It is printed while a
@@ -49,7 +48,6 @@ as fast as possible to navigate through the large search space efficiently and
 find interesting code paths. You should try to get to at least 1,000 exec/s from
 your fuzz target locally before submitting it to the Chromium repository.
 
-
 ### Initialization/Cleanup
 
 Try to keep `LLVMFuzzerTestOneInput` function as simple as possible. If your
@@ -65,7 +63,6 @@ You can skip freeing static resources. However, all resources allocated within
 called millions of times during a fuzzing session. Otherwise, we will hit OOMs
 frequently and reduce overall fuzzing efficiency.
 
-
 ### Memory Usage
 
 Avoid allocation of dynamic memory wherever possible. Memory instrumentation
@@ -74,45 +71,39 @@ works faster for stack-based and static objects, than for heap allocated ones.
 It is always a good idea to try different variants for your fuzz target locally,
 and then submit the fastest implementation.
 
-
 ## Code Coverage
 
-[ClusterFuzz status] page provides source-level coverage report for fuzz targets
-from recent runs. Looking at the report might provide an insight on how to
+[Chrome libFuzzer coverage] provides a source-level coverage report for fuzz
+targets from recent runs. Looking at the report might provide insight on how to
 improve code coverage of a fuzz target.
 
-You can also generate source-level coverage report locally by running the
-[coverage script] stored in Chromium repository. The script provides detailed
-instructions as well as an usage example.
+You can also generate source-level coverage report locally on your particular
+fuzzer by running the [coverage script] stored in Chromium repository. The
+script provides detailed instructions as well as a usage example.
 
 Note that code coverage of a fuzz target **depends heavily** on the corpus
 provided when running the target, i.e. code coverage report generated by a fuzz
 target launched without any corpus would not make much sense.
 
-We encourage you to try out the [coverage script], as it usually generates a better
-code coverage visualization compared to the coverage report hosted on
-ClusterFuzz.
-*NOTE: This is an experimental feature and an active area of work. We are
-working on improving this process.*
-
-
 ## Corpus Size
 
 After running for a while, a fuzz target would reach a plateau and may stop
 discovering new interesting inputs. Corpus for a reasonably complex target
 should contain hundreds (if not thousands) of items.
 
-Too small of a corpus size indicates that fuzz target is hitting a code barrier
-and is unable to get past it. Common cases of such issues include: checksums,
-magic numbers, etc. The easiest way to diagnose this problem is to generate and
-analyze a [coverage report](#Coverage). To fix the issue, you can:
+Too small of a corpus size may indicate that fuzz target is hitting a code
+barrier and is unable to get past it. Common cases of such issues include:
+checksums, magic numbers, etc. However, it also could mean that it is impossible
+for your fuzzer to reach a lot of code. The easiest way to diagnose this problem
+is to generate and analyze a [coverage report](#Code-Coverage). To fix the
+issue, you can:
 
-* Change the code (e.g. disable crc checks while fuzzing), see [Custom Build](#Custom-Build).
+* Change the code (e.g. disable crc checks while fuzzing), see
+  [Custom Build](#Custom-Build).
 * Prepare or improve [seed corpus](#Seed-Corpus).
 * Prepare or improve [fuzzer dictionary](#Fuzzer-Dictionary).
 * Add [custom options](#Custom-Options).
 
-
 ## Seed Corpus
 
 Seed corpus is a set of *valid* and *interesting* inputs that serve as starting
@@ -124,7 +115,8 @@ Seed corpus works especially well for strictly defined file formats and data
 transmission protocols.
 
 * For file format parsers, add valid files from your test suite.
-* For protocol parsers, add valid raw streams from test suite into separate files.
+* For protocol parsers, add valid raw streams from test suite into separate
+  files.
 
 Other examples include a graphics library seed corpus, which would be a variety
 of small PNG/JPG/GIF files.
@@ -145,7 +137,7 @@ ClusterFuzz uses seed corpus defined in Chromium source repository. You need to
 add a `seed_corpus` attribute to your `fuzzer_test` definition in BUILD.gn file:
 
 ```
-fuzzer_test("my_protocol_fuzzer") {
+fuzzer_test("my_fuzzer") {
   ...
   seed_corpus = "test/fuzz/testcases"
   ...
@@ -155,7 +147,7 @@ fuzzer_test("my_protocol_fuzzer") {
 You may specify multiple seed corpus directories via `seed_corpuses` attribute:
 
 ```
-fuzzer_test("my_protocol_fuzzer") {
+fuzzer_test("my_fuzzer") {
   ...
   seed_corpuses = [ "test/fuzz/testcases", "test/unittest/data" ]
   ...
@@ -163,22 +155,28 @@ fuzzer_test("my_protocol_fuzzer") {
 ```
 
 All files found in these directories and their subdirectories will be archived
-into a `<my_fuzzer_name>_seed_corpus.zip` output archive.
+into a `<my_fuzzer>_seed_corpus.zip` output archive.
 
 If you can't store seed corpus in Chromium repository (e.g. it is too large,
 cannot be open sourced, etc), you can upload the corpus to Google Cloud Storage
 bucket used by ClusterFuzz:
 
 1) Go to [Corpus GCS Bucket].
-2) Open directory named `<my_fuzzer_name>_static`. If the directory does not
-exist, please create it.
+2) Open directory named `<my_fuzzer>`. If the directory does not exist,
+please create it.
 3) Upload corpus files into the directory.
 
 Alternative and faster way is to use [gsutil] command line tool:
 ```bash
-gsutil -m rsync <path_to_corpus> gs://clusterfuzz-corpus/libfuzzer/<my_fuzzer_name>_static
+gsutil -m rsync <path_to_corpus> gs://clusterfuzz-corpus/libfuzzer/<my_fuzzer>
 ```
 
+*** note
+**Requirements:** You must have an @google.com and you must be logged into that
+account to write to this bucket (@chromium.org will not work). You can use the
+`gcloud auth login` command to log into your account in `gsutil` if you
+installed `gsutil` through `gcloud`.
+***
 Note that if you upload the corpus to GCS, the `seed_corpus` attribute is not
 needed in your `fuzzer_test` definition.
 
@@ -203,9 +201,6 @@ After running the command above, `seed_corpus_minimized` directory will contain
 a minimized corpus that gives the same code coverage as the initial
 `seed_corpus` directory.
 
-
-
-
 ## Fuzzer Dictionary
 
 It is very useful to provide fuzz target with a set of *common words or values*
@@ -213,8 +208,8 @@ that you expect to find in the input. Adding a dictionary highly improves the
 efficiency of finding new units and works especially well in certain usecases
 (e.g. fuzzing file format decoders or text based protocols like XML).
 
-To add a dictionary, first create a dictionary file. This is a flat text file
-where tokens are listed one per line in the format of `name="value"`, where
+To add a dictionary, first create a dictionary file. This is a flat *ascii* text
+file where tokens are listed one per line in the format of `name="value"`, where
 `name` is optional and can be omitted, although it is a convenient way to
 document the meaning of a particular token. The value must appear in quotes,
 with hex escaping (\xNN) applied to all non-printable, high-bit, or otherwise
@@ -239,7 +234,7 @@ kw3="\xF7\xF8"
 Make sure to test your dictionary by running your fuzz target locally:
 
 ```bash
-./out/libfuzzer/my_protocol_fuzzer -dict=<path_to_dict> <path_to_corpus>
+./out/libfuzzer/my_fuzzer -dict=<path_to_dict> <path_to_corpus>
 ```
 
 If the dictionary is effective, you should see new units discovered in the
@@ -251,38 +246,39 @@ To submit a dictionary to Chromium repository:
 2) Add `dict` attribute to `fuzzer_test` definition in BUILD.gn file:
 
 ```
-fuzzer_test("my_protocol_fuzzer") {
+fuzzer_test("my_fuzzer") {
   ...
-  dict = "my_protocol_fuzzer.dict"
+  dict = "my_fuzzer.dict"
 }
 ```
 
 The dictionary will be used automatically by ClusterFuzz once it picks up a new
 revision build.
 
-
 ## Custom Options
 
 Custom options help to fine tune libFuzzer execution parameters and will also
 override the default values used by ClusterFuzz. Please read [libFuzzer options]
-page for detailed documentation on how these work.
+page for detailed documentation on how these work. A more up-to-date list of
+options can be obtained by using libFuzzer's `-help=1` option (i.e. `./my_fuzzer
+-help=1`).
 
 Add the options needed in `libfuzzer_options` attribute to your `fuzzer_test`
 definition in BUILD.gn file:
 
 ```
-fuzzer_test("my_protocol_fuzzer") {
+fuzzer_test("my_fuzzer") {
   ...
   libfuzzer_options = [
     "max_len=2048",
-    "use_traces=1",
+    "len_control=0",
   ]
 }
 ```
 
-Please note that `dict` parameter should be provided [separately](#Fuzzer-Dictionary).
-All other options can be passed using `libfuzzer_options` property.
-
+Please note that `dict` parameter should be provided
+[separately](#Fuzzer-Dictionary). All other options can be passed using
+`libfuzzer_options` property.
 
 ## Custom Build
 
@@ -294,7 +290,6 @@ fuzz target, but in some cases that might be the only way possible, e.g. when
 there is no intended API to disable checksum verification, or when target code
 uses random generator that affects reproducibility of crashes.
 
-
 [AFL]: http://lcamtuf.coredump.cx/afl/
 [ClusterFuzz status]: clusterfuzz.md#Status-Links
 [Corpus GCS Bucket]: https://console.cloud.google.com/storage/clusterfuzz-corpus/libfuzzer
@@ -303,3 +298,4 @@ uses random generator that affects reproducibility of crashes.
 [gsutil]: https://cloud.google.com/storage/docs/gsutil
 [libFuzzer options]: https://llvm.org/docs/LibFuzzer.html#options
 [startup initialization]: https://llvm.org/docs/LibFuzzer.html#startup-initialization
+[Chrome libFuzzer coverage]: https://chromium-coverage.appspot.com/reports/latest_fuzzers_only/linux/index.html