[WPT/common/security-features] Update README.md

- Move generator-related sections from referrer-policy/README.md
  to common/security-features/README.md
- Add mixed-content/README.md to mention about the generator.
- Update some sections, namely how to remove generated tests, etc.

Bug: 906850
Change-Id: I03f5483435dd9a59b923303d693d7a2cf4717e30
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1785899
Commit-Queue: Hiroshige Hayashizaki <hiroshige@chromium.org>
Reviewed-by: Kenichi Ishibashi <bashi@chromium.org>
Reviewed-by: Hiroki Nakagawa <nhiroki@chromium.org>
Reviewed-by: Mike West <mkwst@chromium.org>
Cr-Commit-Position: refs/heads/master@{#710989}

third_party/blink/web_tests/external/wpt/common/security-features/README.md

-This directory contains the common infrastructure for the following tests.
+
+
+This directory contains the common infrastructure for the following tests (also referred below as projects).
+
 - referrer-policy/
 - mixed-content/
 - upgrade-insecure-requests/
 
 Subdirectories:
+
+- `resources`:
+    Serves JavaScript test helpers.
 - `subresource`:
     Serves subresources, with support for redirects, stash, etc.
     The subresource paths are managed by `subresourceMap` and
@@ -11,10 +17,183 @@ Subdirectories:
 - `scope`:
     Serves nested contexts, such as iframe documents or workers.
     Used from `invokeFrom*()` functions in `resources/common.js`.
+- `tools`:
+    Scripts that generate test HTML files. Not used while running tests.
+- `/referrer-policy/generic/subresource-test`:
+    Sanity checking tests for subresource invocation
+    (This is still placed outside common/)
+
+# Test generator
+
+The test generator (`common/security-features/tools`) generates test HTML files from templates and a seed (`spec.src.json`) that defines all the test scenarios.
+
+The project (i.e. a WPT subdirectory, for example `referrer-policy/`) that uses the generator should define per-project data and invoke the common generator logic in `common/security-features/tools`.
+
+This is the overview of the project structure:
+
+```
+common/security-features/
+└── tools/ - the common test generator logic
+    └── template/ - the test files templates
+project-directory/ (e.g. referrer-policy/)
+├── spec.src.json
+├── generic/
+│   ├── test-case.sub.js - Per-project test helper
+│   └── tools/
+│       └── generator.py - Per-project generator script
+└── gen/ - generated tests
+```
+
+Invoking `project-directory/generic/tools/generate.py` will parse the spec JSON and determine which tests to generate (or skip) while using templates.
+
+## Generating the tests
+
+The repository already contains generated tests, so if you're making changes, see the [Removing all generated tests](#removing-all-generated-tests) section below, on how to remove them before you start generating tests which include your changes.
+
+```bash
+# Chdir into the project directory.
+cd ~/web-platform-tests/project-directory
+
+# Generate the test files under gen/ (HTMLs and .headers files).
+./generic/tools/generate.py
+
+# Add all generated tests to the repo.
+git add gen/ && git commit -m "Add generated tests"
+```
+
+During the generation, the spec is validated by ```common/security-features/tools/spec_validator.py```. This is specially important when you're making changes to  `spec.src.json`. Make sure it's a valid JSON (no comments or trailing commas). The validator reports specific errors (missing keys etc.), if any.
+
+### Removing all generated tests
+
+Simply remove all files under `project-directory/gen/`.
+
+```bash
+# Chdir into the project directory.
+cd ~/web-platform-tests/project-directory
+
+# Remove all generated tests.
+rm -r gen/
+```
+
+### Options for generating tests
+
+Note: this section is currently obsolete. Only the release template is working.
+
+The generator script ```./generic/tools/generate.py``` has two targets: ```release``` and ```debug```.
+
+* Using **release** for the target will produce tests using a template for optimizing size and performance. The release template is intended for the official web-platform-tests and possibly other test suites. No sanity checking is done in release mode. Use this option whenever you're checking into web-platform-tests.
+
+* When generating for ```debug```, the produced tests will contain more verbosity and sanity checks. Use this target to identify problems with the test suites when making changes locally. Make sure you don't check in tests generated with the debug target.
+
+Note that **release** is the default target when invoking ```generate.py```.
+
+
+## Updating the tests
+
+The main test logic lives in ```project-directory/generic/test-case.sub.js``` with helper functions defined in ```/common/security-features/resources/common.js``` so you should probably start there.
+
+For updating the test suites you will most likely do **a subset** of the following:
+
+* Add a new subresource type:
+
+  * Add a new sub-resource python script to `/common/security-features/subresource/`.
+  * Add a sanity check test for a sub-resource to `referrer-policy/generic/subresource-test/`.
+  * Add a new entry to `subresourceMap` in `/common/security-features/resources/common.js`.
+  * Add a new entry to `valid_subresource_names` in `/common/security-features/tools/spec_validator.py`.
+  * Add a new entry to `subresource_schema` in `spec.src.json`.
+  * Update `source_context_schema` to specify in which source context the subresource can be used.
+
+* Add a new subresource redirection type
+
+  * TODO: to be documented. Example: [https://github.com/web-platform-tests/wpt/pull/18939](https://github.com/web-platform-tests/wpt/pull/18939)
+
+* Add a new subresource origin type
+
+  * TODO: to be documented. Example: [https://github.com/web-platform-tests/wpt/pull/18940](https://github.com/web-platform-tests/wpt/pull/18940)
+
+* Add a new source context (e.g. "module sharedworker global scope")
+
+  * TODO: to be documented. Example: [https://github.com/web-platform-tests/wpt/pull/18904](https://github.com/web-platform-tests/wpt/pull/18904)
+
+* Add a new source context list (e.g. "subresource request from a dedicated worker in a `<iframe srcdoc>`")
+
+  * TODO: to be documented.
+
+* Implement new or update existing assertions in ```project-directory/generic/test-case.sub.js```.
+
+* Exclude or add some tests by updating ```spec.src.json``` test expansions.
 
-# spec.src.json format
+* Implement a new delivery method.
 
-## Source Contexts
+  * TODO: to be documented. Currently the support for delivery methods are implemented in many places across `common/security-features/`.
+
+* Regenerate the tests and MANIFEST.json
+
+
+## The spec JSON format
+
+For examples of spec JSON files, see [referrer-policy/spec.src.json](../../referrer-policy/spec.src.json) or  [mixed-content/spec.src.json](../../mixed-content/spec.src.json).
+
+### Main sections
+
+* **`specification`**
+
+  Top level requirements with description fields and a ```test_expansion``` rule.
+  This is closely mimicking the [Referrer Policy specification](http://w3c.github.io/webappsec/specs/referrer-policy/) structure.
+
+* **`excluded_tests`**
+
+  List of ```test_expansion``` patterns expanding into selections which get skipped when generating the tests (aka. blacklisting/suppressing)
+
+* **`test_expansion_schema`**
+
+  Provides valid values for each field.
+  Each test expansion can only contain fields and values defined by this schema (or `"*"` values that indicate all the valid values defined this schema).
+
+* **`subresource_schema`**
+
+  Provides metadata of subresources, e.g. supported delivery types for each subresource.
+
+* **`source_context_schema`**
+
+  Provides metadata of each single source context, e.g. supported delivery types and subresources that can be sent from the context.
+
+* **`source_context_list_schema`**
+
+  Provides possible nested combinations of source contexts. See [Source Contexts](#source-contexts) section below for details.
+
+### Test Expansion Patterns
+
+Each field in a test expansion can be in one of the following formats:
+
+* Single match: ```"value"```
+
+* Match any of: ```["value1", "value2", ...]```
+
+* Match all: ```"*"```
+
+
+**NOTE:** An expansion is always constructive (inclusive), there isn't a negation operator for explicit exclusion. Be aware that using an empty list ```[]``` matches (expands into) exactly nothing. Tests which are to be excluded should be defined in the ```excluded_tests``` section instead.
+
+A single test expansion pattern, be it a requirement or a suppressed pattern, gets expanded into a list of **selections** as follows:
+
+* Expand each field's pattern (single, any of, or all) to list of allowed values (defined by the ```test_expansion_schema```)
+
+* Permute - Recursively enumerate all **selections** across all fields
+
+Be aware that if there is more than one pattern expanding into a same selection, the pattern appearing later in the spec JSON will overwrite a previously generated selection. To make sure this is not undetected when generating, set the value of the ```expansion``` field to ```default``` for an expansion appearing earlier and ```override``` for the one appearing later.
+
+A **selection** is a single **test instance** (scenario) with explicit values that defines a single test. The scenario is then evaluated by the ```TestCase``` in JS. For the rest of the arranging part, examine ```/common/security-features/tools/generate.py``` to see how the values for the templates are produced.
+
+
+Taking the spec JSON, the generator follows this algorithm:
+
+* Expand all ```excluded_tests``` to create a blacklist of selections
+
+* For each specification requirement: Expand the ```test_expansion``` pattern into selections and check each against the blacklist, if not marked as suppresed, generate the test resources for the selection
+
+
+### Source Contexts
 
 In **`source_context_list_schema`**, we can specify
 
@@ -41,7 +220,7 @@ The keys of `source_context_list_schema` can be used as the values of
 `source_context_list` fields, to indicate which source context configuration
 to be used.
 
-## PolicyDelivery placeholders
+### PolicyDelivery placeholders
 
 Each test contains
 
@@ -67,27 +246,51 @@ The following placeholder strings in `source_context_list_schema` can be used:
       the target policy delivery.
 
 For example, when the target policy delivery is
-{deliveryType: "http-rp", key: "referrerPolicy", value: "no-referrer"},
+`{deliveryType: "http-rp", key: "referrerPolicy", value: "no-referrer"}`,
 
-    "sourceContextList": [
-      {"sourceContextType": "top", "policyDeliveries": ["anotherPolicy"]},
-      {"sourceContextType": "classic-worker", "policyDeliveries": ["policy"]}
+```json
+"sourceContextList": [
+  {
+    "sourceContextType": "top",
+    "policyDeliveries": [
+      "anotherPolicy"
+    ]
+  },
+  {
+    "sourceContextType": "classic-worker",
+    "policyDeliveries": [
+      "policy"
     ]
+  }
+]
+```
 
 is replaced with
 
-    "sourceContextList": [
-      {"sourceContextType": "top", "policyDeliveries": [
-          {"deliveryType": "meta",
+```json
+"sourceContextList": [
+  {
+    "sourceContextType": "top",
+    "policyDeliveries": [
+      {
+        "deliveryType": "meta",
         "key": "referrerPolicy",
-           "value": "unsafe-url"}]
+        "value": "unsafe-url"
+      }
+    ]
   },
-      {"sourceContextType": "classic-worker", "policyDeliveries": [
-          {"deliveryType": "http-rp",
+  {
+    "sourceContextType": "classic-worker",
+    "policyDeliveries": [
+      {
+        "deliveryType": "http-rp",
         "key": "referrerPolicy",
-           "value": "no-referrer"}]
+        "value": "no-referrer"
       }
     ]
+  }
+]
+```
 
 which indicates
 
@@ -95,7 +298,7 @@ which indicates
 - The classic worker is created with
   `Referrer-Policy: no-referrer` HTTP response headers.
 
-## `source_context_schema` and `subresource_schema`
+### `source_context_schema` and `subresource_schema`
 
 These represent supported delivery types and subresources
 for each source context or subresource type. These are used

third_party/blink/web_tests/external/wpt/mixed-content/README.md

+# Mixed-content Web Platform Tests
+
+The subdirectory `gen/` is generated using the generator at `common/security-features`.
+See [common/security-features/README.md](../common/security-features/README.md) for how to generate tests.

third_party/blink/web_tests/external/wpt/referrer-policy/README.md

@@ -2,239 +2,5 @@
 
 The Referrer-Policy tests are designed for testing browser implementations and conformance to the [W3 Referrer-Policy Specification](http://w3c.github.io/webappsec/specs/referrer-policy/).
 
-## Project structure
-
-The project contains tools, templates and a seed (```spec.src.json```) for generating tests. The main assertion logic resides in JS files in the root of the ```./generic/``` directory.
-
-This is the overview of the project structure:
-
-```
-.
-└── generic
-    ├── subresource - documents being served as sub-resources (python scripts)
-    ├── subresource-test - sanity checking tests for resource invocation
-    ├── template - the test files template used for generating the tests
-    └── tools -  for generating and maintaining the test suite
-└── (genereated_tests_for_a_specification_1)
-└── (genereated_tests_for_a_specification_2)
-└── ...
-└── (genereated_tests_for_a_specification_N)
-```
-
-## The spec JSON
-
-The ```spec.src.json``` defines all the test scenarios for the referrer policy.
-
-Invoking ```./generic/tools/generate.py``` will parse the spec JSON and determine which tests to generate (or skip) while using templates.
-
-
-The spec can be validated by running ```./generic/tools/spec_validator.py```. This is specially important when you're making changes to  ```spec.src.json```. Make sure it's a valid JSON (no comments or trailing commas). The validator should be informative and very specific on any issues.
-
-The ```spec.src.json``` file can be formatted by running ```../common/security-features/tools/format_spec_src_json.py```.
-
-For details about the spec JSON, see **Overview of the spec JSON** below.
-
-
-## Generating and running the tests
-
-The repository already contains generated tests, so if you're making changes,
-see the **Removing all generated tests** section below, on how to remove them before you start generating tests which include your changes.
-
-Start from the command line:
-
-```bash
-
-# Chdir into the tests directory.
-cd ~/web-platform-tests/referrer-policy
-
-# Generate the test resources.
-./generic/tools/generate.py
-
-# Add all generated tests to the repo.
-git add * && git commit -m "Add generated tests"
-
-# Regenerate the manifest.
-../wpt manifest
-
-```
-
-Navigate to [http://web-platform.test:8000/tools/runner/index.html](http://web-platform.test:8000/tools/runner/index.html).
-
-Run tests under path: ```/referrer-policy```.
-
-Click start.
-
-
-## Options for generating tests
-
-The generator script ```./generic/tools/generate.py``` has two targets: ```release``` and ```debug```.
-
-* Using **release** for the target will produce tests using a template for optimizing size and performance. The release template is intended for the official web-platform-tests and possibly other test suites. No sanity checking is done in release mode. Use this option whenever you're checking into web-platform-tests.
-
-* When generating for ```debug```, the produced tests will contain more verbosity and sanity checks. Use this target to identify problems with the test suite when making changes locally. Make sure you don't check in tests generated with the debug target.
-
-Note that **release** is the default target when invoking ```generate.py```.
-
-
-## Removing all generated tests
-
-```bash
-# Chdir into the tests directory.
-cd ~/web-platform-tests/referrer-policy
-
-# Remove all generated tests.
-./generic/tools/clean.py
-
-# Remove all generated tests to the repo.
-git add * && git commit -m "Remove generated tests"
-
-# Regenerate the manifest.
-../wpt manifest
-```
-
-**Important:**
-The ```./generic/tools/clean.py``` utility will only work if there is a valid ```spec.src.json``` and previously generated directories match the specification requirement names. So make sure you run ```clean.py``` before you alter the specification section of the spec JSON.
-
-
-## Updating the tests
-
-The main test logic lives in ```./generic/referrer-policy-test-case.sub.js``` with helper functions defined in ```./common/security-features/resources/common.js``` so you should probably start there.
-
-For updating the test suite you will most likely do **a subset** of the following:
-
-* Add a new sub-resource python script to ```./generic/subresource/```,
-  and update the reference to it in ```spec.src.json```.
-
-* Add a sanity check test for a sub-resource to ```./generic/subresource-test/```.
-
-* Implement new or update existing assertions in ```./generic/referrer-policy-test-case.sub.js```.
-
-* Exclude or add some tests by updating ```spec.src.json``` test expansions.
-
-* Update the template files living in ```./generic/template/```.
-
-* Implement a new delivery method via HTTP headers or as part of the test template in  ```./generic/tools/generate.py```
-
-* Update the spec schema by editing ```spec.src.json``` while updating the
-  ```./generic/tools/spec_validator.py``` and ```./generic/tools/generate.py```
-  and making sure both still work after the change (by running them).
-
-* Regenerate the tests and MANIFEST.json
-
-
-## Updating the spec and regenerating
-
-When updating the ```spec.src.json```, e.g. by adding a test expansion pattern to the ```excluded_tests``` section or when removing an expansion in the ```specification``` section, make sure to remove all previously generated files which would still get picked up by ```MANIFEST.json``` in the web-platform-tests root. As long as you don't change the specification requirements' names or remove them, you can easily regenerate the tests via command line:
-
-```bash
-
-# Chdir into the tests directory.
-cd ~/web-platform-tests/referrer-policy
-
-# Regenerate the test resources.
-./generic/tools/regenerate
-
-# Add all the tests to the repo.
-git add * && git commit -m "Update generated tests"
-
-# Regenerate the manifest.
-../wpt manifest
-
-
-```
-
-
-## Overview of the spec JSON
-
-**Main sections:**
-
-* **specification**
-
-  Top level requirements with description fields and a ```test_expansion``` rule.
-  This is closely mimicking the [Referrer Policy specification](http://w3c.github.io/webappsec/specs/referrer-policy/) structure.
-
-* **excluded_tests**
-
-  List of ```test_expansion``` patterns expanding into selections which get skipped when generating the tests (aka. blacklisting/suppressing)
-
-* **referrer_policy_schema**
-
-  The schema to validate fields which define the ```referrer_policy``` elsewhere in the JSON.
-  A value for a referrer_policy can only be one specified in the referrer_policy_schema.
-
-* **test_expansion_schema**
-
-  The schema used to check if a ```test_expansion``` is valid.
-  Each test expansion can only contain fields defined by this schema.
-
-### Test Expansion Patterns
-
-Each field in a test expansion can be in one of the following formats:
-
-* Single match: ```"value"```
-
-* Match any of: ```["value1", "value2", ...]```
-
-* Match all: ```"*"```
-
-#### Example: test expansion in a requirement specification
-
-The following example shows how to restrict the expansion of ```referrer_url``` to  ```origin``` and allow rest of the arrangement to expand (permute) to all possible values. The name field will be the prefix of a generated HTML file name for the test.
-
-```json
-    {
-      "name": "origin-only",
-      "title": "Referrer Policy is set to 'origin-only'",
-      "description": "Check that all sub-resources in all cases get only the origin portion of the referrer URL.",
-      "specification_url": "https://w3c.github.io/webappsec/specs/referrer-policy/#referrer-policy-state-origin",
-      "referrer_policy": "origin",
-      "test_expansion": [
-        {
-          "name": "generic",
-          "expansion": "default",
-          "source_protocol": "*",
-          "target_protocol": "*",
-          "delivery_method": "*",
-          "redirection": "*",
-          "origin": "*",
-          "subresource": "*",
-          "referrer_url": "origin"
-        }
-      ]
-    }
-```
-
-**NOTE:** An expansion is always constructive (inclusive), there isn't a negation operator for explicit exclusion. Be aware that using an empty list ```[]``` matches (expands into) exactly nothing. Tests which are to be excluded should be defined in the ```excluded_tests``` section instead.
-
-A single test expansion pattern, be it a requirement or a suppressed pattern, gets expanded into a list of **selections** as follows:
-
-* Expand each field's pattern (single, any of, or all) to list of allowed values (defined by the ```test_expansion_schema```)
-
-* Permute - Recursively enumerate all **selections** across all fields
-
-Be aware that if there is more than one pattern expanding into a same selection (which also shares the same ```name``` field), the pattern appearing later in the spec JSON will overwrite a previously generated selection. To make sure this is not undetected when generating, set the value of the ```expansion``` field to ```default``` for an expansion appearing earlier and ```override``` for the one appearing later.
-
-A **selection** is a single **test instance** (scenario) with explicit values, for example:
-
-```javascript
-var scenario = {
-  "referrer_policy": "origin-when-cross-origin",
-  "delivery_method": "meta-referrer",
-  "redirection": "no-redirect",
-  "origin": "cross-origin",
-  "source_protocol": "http",
-  "target_protocol": "http",
-  "subresource": "iframe-tag",
-  "referrer_url": "origin"
-};
-```
-
-Essentially, this is what gets generated and defines a single test. The scenario is then evaluated by the ```ReferrerPolicyTestCase``` in JS. For the rest of the arranging part, see the ```./generic/template/``` directory and examine ```./generic/tools/generate.py``` to see how the values for the templates are produced.
-
-
-Taking the spec JSON, the generator follows this algorithm:
-
-* Expand all ```excluded_tests``` to create a blacklist of selections
-
-* For each specification requirement: Expand the ```test_expansion``` pattern into selections and check each against the blacklist, if not marked as suppresed, generate the test resources for the selection
-
+The subdirectory `gen/` is generated using the generator at `common/security-features`.
+See [common/security-features/README.md](../common/security-features/README.md) for how to generate tests.

third_party/blink/web_tests/external/wpt/upgrade-insecure-requests/README.md

+# Upgrade-insecure-requests Web Platform Tests
+
+The subdirectory `gen/` is generated using the generator at `common/security-features`.
+See [common/security-features/README.md](../common/security-features/README.md) for how to generate tests.