Update traffic annotation tests documentations.

README.md files in tools/traffic_annotation are updated.

Bug: 690323
Change-Id: Ida0be2dac2f19465650407385d0da66d71277fc1
Reviewed-on: https://chromium-review.googlesource.com/964145
Commit-Queue: Ramin Halavati <rhalavati@chromium.org>
Reviewed-by: Martin Šrámek <msramek@chromium.org>
Cr-Commit-Position: refs/heads/master@{#548701}

tools/traffic_annotation/README.md

-# Running the traffic annotation checkers
-
-The traffic annotation checkers ensure that every operation in the
-code base that talks to the network is properly annotated in the
-source code, so that we can produce reports of what Chromium talks to
-over the network and why.
-Please see `docs/network_traffic_annotations.md` for an introduction to network
-traffic annotations.
-
-To run the checkers, you need a populated build directory, and then
-you do:
-
-```
-$ python tools/annotation_checker/presubmit_checks.py --build-path out/Default
-```
-
-## Building the annotation checker.
-
-The annotation checkers are built as Clang tools. We do not want every
-developer to have to build clang, and so we store pre-built binaries
-in a Google Cloud Storage bucket and retrieve them via gclient hooks.
-
-To roll new versions of the binaries, assuming you have write access
-to the chromium-tools-traffic_annotation bucket, run:
-
-# On Linux:
-```bash
-git new-branch roll_traffic_annotation_tools
-python tools/clang/scripts/update.py --bootstrap --force-local-build \
-    --without-android --extra-tools traffic_annotation_extractor
-cp third_party/llvm-build/Release+Asserts/bin/traffic_annotation_extractor \
-    tools/traffic_annotation/bin/linux64/
-
-# These GN flags produce an optimized, stripped binary that has no dependency
-# on glib.
-gn gen --args='is_official_build=true use_ozone=true' out/Default
-
-ninja -C out/Default traffic_annotation_auditor
-cp -p out/Default/traffic_annotation_auditor \
-    tools/traffic_annotation/bin/linux64
-
-strip tools/traffic_annotation/bin/linux64/traffic_annotation_{auditor,extractor}
-
-third_party/depot_tools/upload_to_google_storage.py \
-    -b chromium-tools-traffic_annotation \
-    tools/traffic_annotation/bin/linux64/traffic_annotation_{auditor,extractor}
-sed -i '/^CLANG_REVISION =/d' tools/traffic_annotation/README.md
-sed -i '/^LASTCHANGE=/d' tools/traffic_annotation/README.md
-grep '^CLANG_REVISION =' tools/clang/scripts/update.py >> tools/traffic_annotation/README.md
-cat build/util/LASTCHANGE >> tools/traffic_annotation/README.md
-git commit -a -m 'Roll traffic_annotation checkers'
-git cl upload
-
-```
-
-# On Windows:
-```bash
-git new-branch roll_traffic_annotation_tools
-python tools/clang/scripts/update.py --bootstrap --force-local-build ^
-    --without-android --extra-tools traffic_annotation_extractor
-cp third_party/llvm-build/Release+Asserts/bin/traffic_annotation_extractor.exe ^
-    tools/traffic_annotation/bin/win32/
-
-# These GN flags produce an optimized, stripped binary that has no dependency
-# on glib.
-gn gen --args="is_official_build=true" out/Default
-
-ninja -C out/Default traffic_annotation_auditor
-cp -p out/Default/traffic_annotation_auditor.exe ^
-    tools/traffic_annotation/bin/win32
-
-python third_party/depot_tools/upload_to_google_storage.py ^
-    -b chromium-tools-traffic_annotation ^
-    tools/traffic_annotation/bin/win32/traffic_annotation_auditor.exe
-python third_party/depot_tools/upload_to_google_storage.py ^
-    -b chromium-tools-traffic_annotation ^
-    tools/traffic_annotation/bin/win32/traffic_annotation_extractor.exe
-sed -i "/^CLANG_REVISION =/d" tools/traffic_annotation/README.md
-sed -i "/^LASTCHANGE=/d" tools/traffic_annotation/README.md
-grep "^CLANG_REVISION =" tools/clang/scripts/update.py >> tools/traffic_annotation/README.md
-cat build/util/LASTCHANGE >> tools/traffic_annotation/README.md
-dos2unix tools/traffic_annotation/README.md
-git commit -a -m 'Roll traffic_annotation checkers'
-git cl upload
-
-```
-
-and land the resulting CL.
-
-The following two lines will be updated by the above script, and the modified
-README should be committed along with the updated .sha1 checksums.
-
-In the event that clang changes something that requires this tool to be
-rebuilt (or for some other reason the tests don't work correctly), please
-disable this test by setting the `TEST_IS_ENABLED` flag to False in
-//tools/traffic_annotation/scripts_check_annotation.py, and file a bug
-and cc the people listed in OWNERS; they'll be on the hook to rebuild and
-re-enable the test.
-
-CLANG_REVISION = '328716'
-LASTCHANGE=71d65441b6c5409be898a817066b60d063380aa9-refs/heads/master@{#547990}
+# Network Traffic Annotations
+Network traffic annotations provide transparency and auditability for the data
+that Chrome sends to the network. For an introduction, please see
+`docs/network_traffic_annotations.md`.
+This folder provides tools to ensure that every operation in the code base that
+requires annotation, is annotated, and annotations are sound and complete.
+
+# Traffic Annotation Auditor
+This is the main executable for all the tests. It runs Traffic Annotation
+Extractor clang tool to check the repository, extract annotations, and perform
+required tests and maintanance. See more details in
+`tools/traffic_annotation/auditor/README.md`.
+
+# Traffic Annotation Extractor
+Traffic Annotation Auditor uses this clang tool (located in
+`tools\clang\traffic_annotation_extractor`) to parse the code and extract
+required data for testing and maintanance.
+
+# Building the Checkers
+We do not want every developer to have to build clang tool and auditor, and so
+we store pre-built binaries in a Google Cloud Storage bucket and retrieve them
+via gclient hooks. The binaries are in `tools/traffic_annotation/bin/[platform]`
+folder. To roll new versions of the binaries, please see
+`tools/traffic_annotation/bin/README.md`.
+
+# Automatic Annotation Tests
+Network traffic annotations are tested in commit queue using
+`tools/traffic_annotation/scripts/check_annotations.py`. This test is currently
+run on Linux and Windows trybots, but may expand in future to other platfroms.
+To perform this test fast enough for a trybot and to avoid spamming the commit
+queue if an unexpected general failure happens (see next item), trybot tests are
+run in error resilient mode and only on the changed files. A more complete test
+runs on an FYI bot using
+`tools/traffic_annotation/scripts/traffic_annotation_auditor_tests.py` and
+alerts if tests are not running as expected.
+
+# Emergency Brake
+In the event that clang changes something that requires the tool to be rebuilt
+(or for some other reason the tests don't work correctly), please disable the
+trybot test by setting the `TEST_IS_ENABLED` flag to False in
+`tools/traffic_annotation/scripts/check_annotations.py`, and file a bug and cc
+the people listed in OWNERS; they'll be on the hook to rebuild and re-enable the
+test.
+
+# Annotations Summary
+`tools/traffic_annotation/summary/annotations.xml` keeps an up to date summary
+of all annotations in the repository. This file is automatically updated by
+Traffic Annotation Auditor.

tools/traffic_annotation/auditor/README.md

 # Network Traffic Annotation Auditor
-This executable runs the clang tool for extraction of Network Traffic
-Annotations from chromium source code and collects and summarizes its outputs.
+This binary runs the clang tool for extraction of Network Traffic Annotations
+from chromium source code, collects and summarizes its outputs, and performs
+tests and maintanance.
 Please see `docs/network_traffic_annotations.md` for an introduction to network
 traffic annotations.
 
 ## Usage
-`traffic_annotation_auditor [OPTION]... [path_filter]...`
+`traffic_annotation_auditor [OPTIONS]... [path_filter]...`
 
-Extracts network traffic annotations from source files. If path filter(s) are
+Extracts network traffic annotations from source files, tests them, and updates
+`tools/traffic_annotation/summary/annotations.xml`. If path filter(s) are
 specified, only those directories of the source will be analyzed.
 Run `traffic_annotation_auditor --help` for options.
 
@@ -15,8 +17,27 @@ Example:
   `traffic_annotation_auditor --build-dir=out/Debug`
 
 The binaries of this file and the clang tool are checked out into
-`tools/traffic_annotation/bin/[platform]`. This is only done for Linux platform
-now and will be extended to other platforms later.
+`tools/traffic_annotation/bin/[platform]`. This is only done for Linux and
+Windows platforms now and will be extended to other platforms later.
 
 ## Running
-Before running, you need to build the COMPLETE chromium.
+Before running, you need to build the COMPLETE chromium and pass the build path
+to the executable.
+
+## Safe List
+If there are files, paths, or specific functions that need to be exempted from
+all or some tests, they can be added to the `safe_list.txt`. The file is comma
+separated, specifying the safe lists based on
+`AuditorException::ExceptionType` in
+`tools/traffic_annotation/auditor/traffic_annotation_auditor.h`.
+Use * as wildcard for zero or more characters when specifying file paths.
+
+Here are the exception types:
+* `all`: Files and paths in this category are excempted from all tests.
+* `missing`: Files and paths in this category can use the
+  MISSING_TRAFFIC_ANNOTATION tag.
+* `direct_assignment`: The functions in this category can assign a value
+  directly to a MutableNetworkTrafficAnnotationTag. This is controlled to
+  avoid assigning arbitrary values to mutable annotations. The format of the
+  entries should be `full_function_name@file_path`. For example:
+  `download::ProtoConversions::EntryFromProto@components/download/internal/background_service/proto_conversions.cc`
\ No newline at end of file

tools/traffic_annotation/auditor/safe_list.txt

 # This is a comma separated file, specifying the safe list for network traffic
-# anntotation auditor. Refer to AuditorException::ExceptionType in
-# 'tools/traffic_annotation/auditor/traffic_annotation_auditor.h' for types of
-# exceptions.
-# Use * as wildcard for zero or more characters.
+# anntotation auditor. Please refer to README.md for more details.
 all,tools/*
 missing,net/url_request/url_fetcher.cc
 missing,net/url_request/url_request_context.cc

tools/traffic_annotation/bin/README.md

+## Building the annotation checker.
+We do not want every developer to have to build clang, and so we store pre-built
+binaries in a Google Cloud Storage bucket and retrieve them via gclient hooks.
+
+To roll new versions of the binaries, assuming you have write access
+to the chromium-tools-traffic_annotation bucket, run:
+
+# On Linux:
+```bash
+git new-branch roll_traffic_annotation_tools
+python tools/clang/scripts/update.py --bootstrap --force-local-build \
+    --without-android --extra-tools traffic_annotation_extractor
+cp third_party/llvm-build/Release+Asserts/bin/traffic_annotation_extractor \
+    tools/traffic_annotation/bin/linux64/
+
+# These GN flags produce an optimized, stripped binary that has no dependency
+# on glib.
+gn gen --args='is_official_build=true use_ozone=true' out/Default
+
+ninja -C out/Default traffic_annotation_auditor
+cp -p out/Default/traffic_annotation_auditor \
+    tools/traffic_annotation/bin/linux64
+
+strip tools/traffic_annotation/bin/linux64/traffic_annotation_{auditor,extractor}
+
+third_party/depot_tools/upload_to_google_storage.py \
+    -b chromium-tools-traffic_annotation \
+    tools/traffic_annotation/bin/linux64/traffic_annotation_{auditor,extractor}
+sed -i '/^CLANG_REVISION =/d' tools/traffic_annotation/bin/README.md
+sed -i '/^LASTCHANGE=/d' tools/traffic_annotation/bin/README.md
+grep '^CLANG_REVISION =' tools/clang/scripts/update.py >> tools/traffic_annotation/bin/README.md
+cat build/util/LASTCHANGE >> tools/traffic_annotation/bin/README.md
+git commit -a -m 'Roll traffic_annotation checkers'
+git cl upload
+
+```
+
+# On Windows:
+```bash
+git new-branch roll_traffic_annotation_tools
+python tools/clang/scripts/update.py --bootstrap --force-local-build ^
+    --without-android --extra-tools traffic_annotation_extractor
+cp third_party/llvm-build/Release+Asserts/bin/traffic_annotation_extractor.exe ^
+    tools/traffic_annotation/bin/win32/
+
+# These GN flags produce an optimized, stripped binary that has no dependency
+# on glib.
+gn gen --args="is_official_build=true" out/Default
+
+ninja -C out/Default traffic_annotation_auditor
+cp -p out/Default/traffic_annotation_auditor.exe ^
+    tools/traffic_annotation/bin/win32
+
+python third_party/depot_tools/upload_to_google_storage.py ^
+    -b chromium-tools-traffic_annotation ^
+    tools/traffic_annotation/bin/win32/traffic_annotation_auditor.exe
+python third_party/depot_tools/upload_to_google_storage.py ^
+    -b chromium-tools-traffic_annotation ^
+    tools/traffic_annotation/bin/win32/traffic_annotation_extractor.exe
+sed -i "/^CLANG_REVISION =/d" tools/traffic_annotation/bin/README.md
+sed -i "/^LASTCHANGE=/d" tools/traffic_annotation/bin/README.md
+grep "^CLANG_REVISION =" tools/clang/scripts/update.py >> ^
+    tools/traffic_annotation/bin/README.md
+cat build/util/LASTCHANGE >> tools/traffic_annotation/bin/README.md
+dos2unix tools/traffic_annotation/bin/README.md
+git commit -a -m 'Roll traffic_annotation checkers'
+git cl upload
+
+```
+
+and land the resulting CL.
+
+The following two lines will be updated by the above script, and the modified
+README should be committed along with the updated .sha1 checksums.
+
+CLANG_REVISION = '328716'
+LASTCHANGE=71d65441b6c5409be898a817066b60d063380aa9-refs/heads/master@{#547990}
\ No newline at end of file

tools/traffic_annotation/scripts/README.md

 # Traffic Annotation Scripts
-This file describes the scripts in `tools/traffic_annotation/scripts`
+This file describes the scripts in `tools/traffic_annotation/scripts`.
 
 
 # check_annotations.py

tools/traffic_annotation/summary/README.md

@@ -5,20 +5,30 @@ traffic annotations.
 
 # Content Description
 `annotations.xml` includes the summary of all network traffic annotations in
-Chromium repository. The content includes complete annotations and the merged
-partial and completing (and branched completing) annotations.
-For each annotation, unique id, hash code of unique id, hash code of the
-content, and the list of OSes using this annotation is presented. If annotation
-is a reserved one, instead of content hash code, a `reserved` attribute is
-included.
-Once an annotation is removed from the repository, a `deprecated` attribute is
-added to its item in this file, with value equal to the deprecation date.
-These items can be manually or automatically pruned after sufficient time.
+Chromium repository.
+The following items are stored for each annotation :
+* `id`: Unique ID of the annotation.
+* `hash_code`: Hash code of the unique id of the annotation. These values are
+     used in the binary as annotation tags.
+* `type`: Type of the annotation (complete, partial, ...). Uses enum values
+    of `AnnotationInstance` in `tools/traffic_annotation/auditor/instance.h`.
+* `content_hash_code`: Hash code of the annotation content. This value is stored
+    to check when an annotation is modified.
+* `os_list`: List of all platforms on which this annotation exists.
+    Currently only including `linux` and `windows`.
+* `file_path`: The file path of the annotation.
+* `reserved`: Reserverd annotations (like annotation for test files) have this
+    attribute. If annotation is a reserverd one, it does not have
+  `content_hash_code` and `file_path` attributes.
+* `deprecated`: Once an annotation is removed from the repository, this
+    attribute is added to its item with value equal to the deprecation date, and
+    `os_list` and `file_path` attributes are removed.
+    These items can be manually or automatically pruned after sufficient time.
+    Unique id of deprecated annotations cannot be reused.
 
 # How to Generate/Update.
 Run `traffic_annotation_auditor` to check for annotations correctness and
-automatic update. After each modification of`annotations.xml`, auditor calls
-`tools/traffic_annotation/scripts/annotations_xml_downstream_updater.py` to
-update all users of this file.
+automatic update. There are also trybots on Linux and Windows to run the tests
+and suggest required updates.
 The latest executable of `traffic_annotation_auditor` for supported platforms
 can be found in `tools/traffic_annotation/bin/[platform]`.