Skip to content

GitLab

T
tangled
Commit 7a052b23 authored by Jonathan Metzman's avatar Jonathan Metzman Committed by Commit Bot
Browse files
Options

[libFuzzer] Improve docs 

Improve efficient_fuzzer.md by making the following changes:
1. Get rid of static corpus instructions (just use regular corpus)
2. Add details about authenticating to use ClusterFuzz buckets.
3. Eliminate extra newlines
4. Be consistent about the example fuzzer name
5. Mention dict should be ascii encoded

Improve clusterfuzz.md by linking directly to coverage page and by
linking to most recent stats.

Change-Id: I9990ab8ab4811445214735e65a6e8d2bd2319179
Reviewed-on: https://chromium-review.googlesource.com/c/1323272
Commit-Queue: Jonathan Metzman <metzman@chromium.org>
Reviewed-by: default avatarMax Moroz <mmoroz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#608455}
parent 2616eba5
Show whitespace changes
Inline Side-by-side
Showing with 45 additions and 48 deletions
testing/libfuzzer/clusterfuzz.md
View file @ 7a052b23
......@@ -8,8 +8,8 @@ Googlers can read more [here](https://goto.google.com/clusterfuzz).
## Status Links
* [Buildbot] - status of all libFuzzer builds.
* [ClusterFuzz Fuzzer Status] - fuzzing metrics, links to crashes and coverage
reports.
* [ClusterFuzz Fuzzer Status] - fuzzing metrics.
* [Code Coverage] - code coverage of libFuzzer targets in Chrome.
* [ClusterFuzz libFuzzer Logs] - individual fuzz target run logs.
* [Corpus GCS Bucket] - current corpus for each fuzz target. Can be used to
upload bootstrapped corpus.
......@@ -46,8 +46,9 @@ goes into general corpus directory. Seed corpus is read-only.
[Buildbot]: https://build.chromium.org/p/chromium.fyi/buildslaves/slave43-c1
[Code Coverage]: https://chromium-coverage.appspot.com/reports/latest_fuzzers_only/linux/index.html
[chromium_libfuzzer.py]: https://code.google.com/p/chromium/codesearch#chromium/build/scripts/slave/recipes/chromium_libfuzzer.py
[ClusterFuzz Fuzzer Status]: https://clusterfuzz.com/fuzzer-stats
[ClusterFuzz Fuzzer Stats]: https://clusterfuzz.com/v2/fuzzer-stats/by-fuzzer/fuzzer/libFuzzer/job/libfuzzer_chrome_asan
[ClusterFuzz libFuzzer Logs]: https://console.cloud.google.com/storage/browser/clusterfuzz-libfuzzer-logs
[Corpus GCS Bucket]: https://console.cloud.google.com/storage/clusterfuzz-corpus/libfuzzer
[fuzzer_test.gni]: https://code.google.com/p/chromium/codesearch#chromium/src/testing/libfuzzer/fuzzer_test.gni
testing/libfuzzer/efficient_fuzzer.md
View file @ 7a052b23
......@@ -34,7 +34,6 @@ There are other ways that are useful in some cases, but not always applicable:
* [Custom Options](#Custom-Options)
* [Custom Build](#Custom-Build)
## Execution Speed
Fuzz target speed is calculated in executions per second. It is printed while a
......@@ -49,7 +48,6 @@ as fast as possible to navigate through the large search space efficiently and
find interesting code paths. You should try to get to at least 1,000 exec/s from
your fuzz target locally before submitting it to the Chromium repository.
### Initialization/Cleanup
Try to keep `LLVMFuzzerTestOneInput` function as simple as possible. If your
......@@ -65,7 +63,6 @@ You can skip freeing static resources. However, all resources allocated within
called millions of times during a fuzzing session. Otherwise, we will hit OOMs
frequently and reduce overall fuzzing efficiency.
### Memory Usage
Avoid allocation of dynamic memory wherever possible. Memory instrumentation
......@@ -74,45 +71,39 @@ works faster for stack-based and static objects, than for heap allocated ones.
It is always a good idea to try different variants for your fuzz target locally,
and then submit the fastest implementation.
## Code Coverage
[ClusterFuzz status] page provides source-level coverage report for fuzz targets
from recent runs. Looking at the report might provide an insight on how to
[Chrome libFuzzer coverage] provides a source-level coverage report for fuzz
targets from recent runs. Looking at the report might provide insight on how to
improve code coverage of a fuzz target.
You can also generate source-level coverage report locally by running the
[coverage script] stored in Chromium repository. The script provides detailed
instructions as well as an usage example.
You can also generate source-level coverage report locally on your particular
fuzzer by running the [coverage script] stored in Chromium repository. The
script provides detailed instructions as well as a usage example.
Note that code coverage of a fuzz target **depends heavily** on the corpus
provided when running the target, i.e. code coverage report generated by a fuzz
target launched without any corpus would not make much sense.
We encourage you to try out the [coverage script], as it usually generates a better
code coverage visualization compared to the coverage report hosted on
ClusterFuzz.
*NOTE: This is an experimental feature and an active area of work. We are
working on improving this process.*
## Corpus Size
After running for a while, a fuzz target would reach a plateau and may stop
discovering new interesting inputs. Corpus for a reasonably complex target
should contain hundreds (if not thousands) of items.
Too small of a corpus size indicates that fuzz target is hitting a code barrier
and is unable to get past it. Common cases of such issues include: checksums,
magic numbers, etc. The easiest way to diagnose this problem is to generate and
analyze a [coverage report](#Coverage). To fix the issue, you can:
Too small of a corpus size may indicate that fuzz target is hitting a code
barrier and is unable to get past it. Common cases of such issues include:
checksums, magic numbers, etc. However, it also could mean that it is impossible
for your fuzzer to reach a lot of code. The easiest way to diagnose this problem
is to generate and analyze a [coverage report](#Code-Coverage). To fix the
issue, you can:
* Change the code (e.g. disable crc checks while fuzzing), see [Custom Build](#Custom-Build).
* Change the code (e.g. disable crc checks while fuzzing), see
[Custom Build](#Custom-Build).
* Prepare or improve [seed corpus](#Seed-Corpus).
* Prepare or improve [fuzzer dictionary](#Fuzzer-Dictionary).
* Add [custom options](#Custom-Options).
## Seed Corpus
Seed corpus is a set of *valid* and *interesting* inputs that serve as starting
......@@ -124,7 +115,8 @@ Seed corpus works especially well for strictly defined file formats and data
transmission protocols.
* For file format parsers, add valid files from your test suite.
* For protocol parsers, add valid raw streams from test suite into separate files.
* For protocol parsers, add valid raw streams from test suite into separate
files.
Other examples include a graphics library seed corpus, which would be a variety
of small PNG/JPG/GIF files.
......@@ -145,7 +137,7 @@ ClusterFuzz uses seed corpus defined in Chromium source repository. You need to
add a `seed_corpus` attribute to your `fuzzer_test` definition in BUILD.gn file:
```
fuzzer_test("my_protocol_fuzzer") {
fuzzer_test("my_fuzzer") {
...
seed_corpus = "test/fuzz/testcases"
...
......@@ -155,7 +147,7 @@ fuzzer_test("my_protocol_fuzzer") {
You may specify multiple seed corpus directories via `seed_corpuses` attribute:
```
fuzzer_test("my_protocol_fuzzer") {
fuzzer_test("my_fuzzer") {
...
seed_corpuses = [ "test/fuzz/testcases", "test/unittest/data" ]
...
......@@ -163,22 +155,28 @@ fuzzer_test("my_protocol_fuzzer") {
```
All files found in these directories and their subdirectories will be archived
into a `<my_fuzzer_name>_seed_corpus.zip` output archive.
into a `<my_fuzzer>_seed_corpus.zip` output archive.
If you can't store seed corpus in Chromium repository (e.g. it is too large,
cannot be open sourced, etc), you can upload the corpus to Google Cloud Storage
bucket used by ClusterFuzz:
1) Go to [Corpus GCS Bucket].
2) Open directory named `<my_fuzzer_name>_static`. If the directory does not
exist, please create it.
2) Open directory named `<my_fuzzer>`. If the directory does not exist,
please create it.
3) Upload corpus files into the directory.
Alternative and faster way is to use [gsutil] command line tool:
```bash
gsutil -m rsync <path_to_corpus> gs://clusterfuzz-corpus/libfuzzer/<my_fuzzer_name>_static
gsutil -m rsync <path_to_corpus> gs://clusterfuzz-corpus/libfuzzer/<my_fuzzer>
```
*** note
**Requirements:** You must have an @google.com and you must be logged into that
account to write to this bucket (@chromium.org will not work). You can use the
`gcloud auth login` command to log into your account in `gsutil` if you
installed `gsutil` through `gcloud`.
***
Note that if you upload the corpus to GCS, the `seed_corpus` attribute is not
needed in your `fuzzer_test` definition.
......@@ -203,9 +201,6 @@ After running the command above, `seed_corpus_minimized` directory will contain
a minimized corpus that gives the same code coverage as the initial
`seed_corpus` directory.
## Fuzzer Dictionary
It is very useful to provide fuzz target with a set of *common words or values*
......@@ -213,8 +208,8 @@ that you expect to find in the input. Adding a dictionary highly improves the
efficiency of finding new units and works especially well in certain usecases
(e.g. fuzzing file format decoders or text based protocols like XML).
To add a dictionary, first create a dictionary file. This is a flat text file
where tokens are listed one per line in the format of `name="value"`, where
To add a dictionary, first create a dictionary file. This is a flat *ascii* text
file where tokens are listed one per line in the format of `name="value"`, where
`name` is optional and can be omitted, although it is a convenient way to
document the meaning of a particular token. The value must appear in quotes,
with hex escaping (\xNN) applied to all non-printable, high-bit, or otherwise
......@@ -239,7 +234,7 @@ kw3="\xF7\xF8"
Make sure to test your dictionary by running your fuzz target locally:
```bash
./out/libfuzzer/my_protocol_fuzzer -dict=<path_to_dict> <path_to_corpus>
./out/libfuzzer/my_fuzzer -dict=<path_to_dict> <path_to_corpus>
```
If the dictionary is effective, you should see new units discovered in the
......@@ -251,38 +246,39 @@ To submit a dictionary to Chromium repository:
2) Add `dict` attribute to `fuzzer_test` definition in BUILD.gn file:
```
fuzzer_test("my_protocol_fuzzer") {
fuzzer_test("my_fuzzer") {
...
dict = "my_protocol_fuzzer.dict"
dict = "my_fuzzer.dict"
}
```
The dictionary will be used automatically by ClusterFuzz once it picks up a new
revision build.
## Custom Options
Custom options help to fine tune libFuzzer execution parameters and will also
override the default values used by ClusterFuzz. Please read [libFuzzer options]
page for detailed documentation on how these work.
page for detailed documentation on how these work. A more up-to-date list of
options can be obtained by using libFuzzer's `-help=1` option (i.e. `./my_fuzzer
-help=1`).
Add the options needed in `libfuzzer_options` attribute to your `fuzzer_test`
definition in BUILD.gn file:
```
fuzzer_test("my_protocol_fuzzer") {
fuzzer_test("my_fuzzer") {
...
libfuzzer_options = [
"max_len=2048",
"use_traces=1",
"len_control=0",
]
}
```
Please note that `dict` parameter should be provided [separately](#Fuzzer-Dictionary).
All other options can be passed using `libfuzzer_options` property.
Please note that `dict` parameter should be provided
[separately](#Fuzzer-Dictionary). All other options can be passed using
`libfuzzer_options` property.
## Custom Build
......@@ -294,7 +290,6 @@ fuzz target, but in some cases that might be the only way possible, e.g. when
there is no intended API to disable checksum verification, or when target code
uses random generator that affects reproducibility of crashes.
[AFL]: http://lcamtuf.coredump.cx/afl/
[ClusterFuzz status]: clusterfuzz.md#Status-Links
[Corpus GCS Bucket]: https://console.cloud.google.com/storage/clusterfuzz-corpus/libfuzzer
......@@ -303,3 +298,4 @@ uses random generator that affects reproducibility of crashes.
[gsutil]: https://cloud.google.com/storage/docs/gsutil
[libFuzzer options]: https://llvm.org/docs/LibFuzzer.html#options
[startup initialization]: https://llvm.org/docs/LibFuzzer.html#startup-initialization
[Chrome libFuzzer coverage]: https://chromium-coverage.appspot.com/reports/latest_fuzzers_only/linux/index.html
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment