Commit af9811f2 authored by Abhishek Arya's avatar Abhishek Arya Committed by Commit Bot

Coverage: doc improvements.

R=mmoroz@chromium.org,liaoyuke@chromium.org

Change-Id: Iaf6f74ab4bc02d95f93f0ffc44ccf20e0d1f0682
Bug: 833912
Reviewed-on: https://chromium-review.googlesource.com/1056153
Commit-Queue: Abhishek Arya <inferno@chromium.org>
Reviewed-by: default avatarMax Moroz <mmoroz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#558035}
parent 89733b60
...@@ -11,12 +11,12 @@ Table of contents: ...@@ -11,12 +11,12 @@ Table of contents:
- [Contacts](#contacts) - [Contacts](#contacts)
- [FAQ](#faq) - [FAQ](#faq)
Chromium uses Clang source-based code coverage, this [documentation] explains Chromium uses Clang source-based code coverage. This [documentation] explains
how to use Clang’s source-based coverage features in general. how to use Clang’s source-based coverage features in general.
In this doc, we first introduce a code coverage script that can be used to In this document, we first introduce a code coverage script that can be used to
generate code coverage reports for Chromium code in one command, and then dive generate code coverage reports for Chromium code in one command, and then
into details to describe the code coverage reports generation workflow. describe the code coverage reports generation workflow.
## Coverage Script ## Coverage Script
The [coverage script] automates the process described below and provides a The [coverage script] automates the process described below and provides a
...@@ -37,22 +37,34 @@ $ python tools/code_coverage/coverage.py \ ...@@ -37,22 +37,34 @@ $ python tools/code_coverage/coverage.py \
-f url/ -f crypto/ -f url/ -f crypto/
``` ```
The command above builds `crypto_unittests` and `url_unittests` targets and then The command above builds `crypto_unittests` and `url_unittests` targets and then
runs each with the command and arguments specified by the `-c` flag. For runs them individually with their commands and arguments specified by the `-c` flag.
`url_unittests`, it only runs the test `URLParser.PathURL`. The coverage report For `url_unittests`, it only runs the test `URLParser.PathURL`. The coverage report
is filtered to include only files and sub-directories under `url/` and `crypto/` is filtered to include only files and sub-directories under `url/` and `crypto/`
directories. directories.
Aside from automating the process, this script provides additional features to Aside from automating the process, this script provides visualization features to
view code coverage breakdown by directories and by components, for example: view code coverage breakdown by directories and by components, for example:
Directory View: ### Directory View
![code coverage report directory view] ![code coverage report directory view]
Component View: ### Component View
![code coverage report component view] ![code coverage report component view]
### Source View
When you click on a particular source file in one of the views above, you can check
per-line coverage information such as
- Uncovered / Covered line fragments, lines and code blocks. This information can be
useful to identify areas of code that lack test coverage.
- Per-line hit counts indicating how many times this line was hit by all tested targets.
This information can be useful to determine hot spots in your code.
![code coverage source view]
## Workflow ## Workflow
This section presents the workflow of generating code coverage reports using two This section presents the workflow of generating code coverage reports using two
unit test targets in Chromium repo as an example: `crypto_unittests` and unit test targets in Chromium repo as an example: `crypto_unittests` and
...@@ -82,11 +94,11 @@ $ ninja -C out/coverage crypto_unittests url_unittests ...@@ -82,11 +94,11 @@ $ ninja -C out/coverage crypto_unittests url_unittests
``` ```
### Step 2 Create Raw Profiles ### Step 2 Create Raw Profiles
The next step is to run the instrumented binaries, and when the program exits it The next step is to run the instrumented binaries. When the program exits, it
will write a raw profile for each process. Because Chromium runs tests in writes a raw profile for each process. Because Chromium runs tests in
multiple processes, and the number of processes spawned can be as many as a few multiple processes, the number of processes spawned can be as many as a few
hundred, which results in the generation of a few hundred gigabytes’ raw hundred, resulting in the generation of a few hundred gigabytes’ raw
profiles, to limit the number of raw profiles, `%Nm` pattern in profiles. To limit the number of raw profiles, `%Nm` pattern in
`LLVM_PROFILE_FILE` environment variable is used to run tests in multi-process `LLVM_PROFILE_FILE` environment variable is used to run tests in multi-process
mode, where `N` is the number of raw profiles. With `N = 4`, the total size of mode, where `N` is the number of raw profiles. With `N = 4`, the total size of
the raw profiles are limited to a few gigabytes. the raw profiles are limited to a few gigabytes.
...@@ -103,10 +115,10 @@ crypto_unittests.3657994905831792357_3.profraw ...@@ -103,10 +115,10 @@ crypto_unittests.3657994905831792357_3.profraw
### Step 3 Create Indexed Profile ### Step 3 Create Indexed Profile
Raw profiles must be indexed before generating code coverage reports, and this Raw profiles must be indexed before generating code coverage reports, and this
is done using the `merge` command of `llvm-profdata` tool, which merges multiple is done using the `merge` command of `llvm-profdata` tool, which merges multiple
raw profiles (.profraw) and index them to create a single profile (.profdata). raw profiles (.profraw) and indexes them to create a single profile (.profdata).
At this point, all the raw profiles can be thrown away because their information At this point, all the raw profiles can be thrown away because their information
are already contained in the indexed profile. is already contained in the indexed profile.
``` ```
$ llvm-profdata merge -o out/report/coverage.profdata \ $ llvm-profdata merge -o out/report/coverage.profdata \
...@@ -122,10 +134,12 @@ out/report/coverage.profdata ...@@ -122,10 +134,12 @@ out/report/coverage.profdata
### Step 4 Create Coverage Reports ### Step 4 Create Coverage Reports
Finally, `llvm-cov` is used to render code coverage reports. There are different Finally, `llvm-cov` is used to render code coverage reports. There are different
report generation modes, and all of them require the indexed profile, all the report generation modes, and all of them require the following as input:
built binaries and all the exercised source files to be available. - Indexed profile
- All built target binaries
- All exercised source files.
For example, following command can be used to generate per-file line-by-line For example, the following command can be used to generate per-file line-by-line
code coverage report: code coverage report:
``` ```
...@@ -156,7 +170,7 @@ see [crbug.com/831939]. ...@@ -156,7 +170,7 @@ see [crbug.com/831939].
### I am getting some warnings while using the script, is that fine? ### I am getting some warnings while using the script, is that fine?
Usually that is not a critical issue, but in general we tend not to have any Usually this is not a critical issue, but in general we tend not to have any
warnings. Please check the list of [known issues], and if there is a similar warnings. Please check the list of [known issues], and if there is a similar
bug, leave a comment with the command you run, the output you get, and Chromium bug, leave a comment with the command you run, the output you get, and Chromium
revision you use. Otherwise, please [file a new issue] providing the same revision you use. Otherwise, please [file a new issue] providing the same
...@@ -168,7 +182,8 @@ If a crash of any type occurs (Segmentation Fault, CHECK failure, ASan error), ...@@ -168,7 +182,8 @@ If a crash of any type occurs (Segmentation Fault, CHECK failure, ASan error),
the crashing process will not dump coverage information necessary to generate the crashing process will not dump coverage information necessary to generate
code coverage report. For single-process applications (e.g. fuzz targets), that code coverage report. For single-process applications (e.g. fuzz targets), that
means no coverage will be reported at all. For multi-process applications, the means no coverage will be reported at all. For multi-process applications, the
report will be incomplete. report will be incomplete. It is important to fix the crash first. If this is
happening only in the coverage instrumented build, please [file a bug].
### Is it possible to obtain code coverage from a full Chromium build? ### Is it possible to obtain code coverage from a full Chromium build?
...@@ -189,6 +204,7 @@ For more information, please see [crbug.com/834781]. ...@@ -189,6 +204,7 @@ For more information, please see [crbug.com/834781].
[coverage script]: https://cs.chromium.org/chromium/src/tools/code_coverage/coverage.py [coverage script]: https://cs.chromium.org/chromium/src/tools/code_coverage/coverage.py
[code coverage report directory view]: images/code_coverage_directory_view.png [code coverage report directory view]: images/code_coverage_directory_view.png
[code coverage report component view]: images/code_coverage_component_view.png [code coverage report component view]: images/code_coverage_component_view.png
[code coverage source view]: images/code_coverage_source_view.png
[crbug.com/831939]: https://crbug.com/831939 [crbug.com/831939]: https://crbug.com/831939
[crbug.com/834781]: https://crbug.com/834781 [crbug.com/834781]: https://crbug.com/834781
[documentation]: https://clang.llvm.org/docs/SourceBasedCodeCoverage.html [documentation]: https://clang.llvm.org/docs/SourceBasedCodeCoverage.html
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment