Commit adb0670a authored by Xianzhu Wang's avatar Xianzhu Wang Committed by Commit Bot

Update docs for runtime enabled features and web tests

Change-Id: I7230bd37e7f9c1c136e1c23b6acdc340eb089121
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2303452Reviewed-by: default avatarPhilip Rogers <pdr@chromium.org>
Commit-Queue: Xianzhu Wang <wangxianzhu@chromium.org>
Cr-Commit-Position: refs/heads/master@{#789247}
parent f98aa5d1
...@@ -217,7 +217,7 @@ on this. ...@@ -217,7 +217,7 @@ on this.
There are two ways to run web tests with additional command-line arguments: There are two ways to run web tests with additional command-line arguments:
* Using `--additional-driver-flag`: * Using `--additional-driver-flag` or `--flag-specific`:
```bash ```bash
third_party/blink/tools/run_web_tests.py --additional-driver-flag=--blocking-repaint third_party/blink/tools/run_web_tests.py --additional-driver-flag=--blocking-repaint
...@@ -300,8 +300,15 @@ There are two ways to run web tests with additional command-line arguments: ...@@ -300,8 +300,15 @@ There are two ways to run web tests with additional command-line arguments:
with the flags without creating any virtual tests. with the flags without creating any virtual tests.
For flags whose implementation is still in progress, virtual test suites and For flags whose implementation is still in progress, virtual test suites and
flag-specific expectations represent two alternative strategies for testing. flag-specific expectations represent two alternative strategies for testing both
Consider the following when choosing between them: the enabled code path and not-enabled code path. They are preferred to only
setting a [runtime enabled feature](../../third_party/blink/renderer/platform/RuntimeEnabledFeatures.md)
to `status: "test"` if the feature has substantially different code path from
production because the latter would cause loss of test coverage of the production
code path.
Consider the following when choosing between virtual test suites and
flag-specific expectations:
* The * The
[waterfall builders](https://dev.chromium.org/developers/testing/chromium-build-infrastructure/tour-of-the-chromium-buildbot) [waterfall builders](https://dev.chromium.org/developers/testing/chromium-build-infrastructure/tour-of-the-chromium-buildbot)
...@@ -309,7 +316,8 @@ Consider the following when choosing between them: ...@@ -309,7 +316,8 @@ Consider the following when choosing between them:
will run all virtual test suites in addition to the non-virtual tests. will run all virtual test suites in addition to the non-virtual tests.
Conversely, a flag-specific expectations file won't automatically cause the Conversely, a flag-specific expectations file won't automatically cause the
bots to test your flag - if you want bot coverage without virtual test suites, bots to test your flag - if you want bot coverage without virtual test suites,
you will need to set up a dedicated bot for your flag. you will need to set up a dedicated bot ([example](https://chromium-review.googlesource.com/c/chromium/src/+/1850255))
for your flag.
* Due to the above, virtual test suites incur a performance penalty for the * Due to the above, virtual test suites incur a performance penalty for the
commit queue and the continuous build infrastructure. This is exacerbated by commit queue and the continuous build infrastructure. This is exacerbated by
......
...@@ -14,7 +14,7 @@ Example: ...@@ -14,7 +14,7 @@ Example:
``` ```
The status of the feature controls when it will be enabled in the Blink engine. The status of the feature controls when it will be enabled in the Blink engine.
| Status Value | Web feature enabled during web tests with content_shell [1] | Web feature enabled as part of web experimental features [2] | Web feature enabled in stable release | Non-web exposed feature enabled through a command line flag [3] | Status Value | Web feature enabled during [web tests] with content_shell [1] | Web feature enabled as part of web experimental features [2] | Web feature enabled in stable release | Non-web exposed feature enabled through a command line flag [3]
|:---:|:---:|:---:|:---:|:---:| |:---:|:---:|:---:|:---:|:---:|
| <missing\> | No | No | No | Yes | | <missing\> | No | No | No | Yes |
| `test` | Yes | No | No | No | | `test` | Yes | No | No | No |
...@@ -134,16 +134,17 @@ interface ExistingObject { ...@@ -134,16 +134,17 @@ interface ExistingObject {
**Warning:** You will not be able to change the enabled state of these at runtime as the V8 object templates definitions are created during start up and will not be updated during runtime. **Warning:** You will not be able to change the enabled state of these at runtime as the V8 object templates definitions are created during start up and will not be updated during runtime.
## Web Tests (JavaScript) ## Web Tests (JavaScript)
Test whether a feature is enabled using:
In [web tests], you can test whether a feature is enabled using:
```javascript ```javascript
internals.runtimeFlags.amazingNewFeatureEnabled internals.runtimeFlags.amazingNewFeatureEnabled
``` ```
This attribute is read only and cannot be changed. This attribute is read only and cannot be changed, unless `settable_from_internals: true` is specified for the feature.
**Note:** The `internals` JavaScript API is only available in ContentShell for use by web tests and does not appear in released versions of Chromium. **Note:** The `internals` JavaScript API is only available in content_shell for use by web tests and does not appear in Chromium. In content_shell's browser mode, `--expose-internals-for-testing` is needed to have the `internals` JavaScript API.
### Running Web Tests ### Running Web Tests
When content_shell is run with `--stable-release-mode` flag, test-only features (ones listed in [runtime_enabled_features.json5] with `test`) are turned off. When content_shell is run for web tests with `--stable-release-mode` flag, test-only and experimental features (ones listed in [runtime_enabled_features.json5] with `status: "test"` or `status: "experimental"`) are turned off. The [virtual/stable] suite runs with the flag, which is one of the ways to ensure test coverage of production code path for these features.
## Generated Files ## Generated Files
[renderer/build/scripts/make_runtime_features.py][make_runtime_features.py] uses [runtime_enabled_features.json5] to generate: [renderer/build/scripts/make_runtime_features.py][make_runtime_features.py] uses [runtime_enabled_features.json5] to generate:
...@@ -173,7 +174,7 @@ After applying most other feature settings, the features requested feature setti ...@@ -173,7 +174,7 @@ After applying most other feature settings, the features requested feature setti
https://groups.google.com/a/chromium.org/d/msg/blink-dev/JBakhu5J6Qs/re2LkfEslTAJ https://groups.google.com/a/chromium.org/d/msg/blink-dev/JBakhu5J6Qs/re2LkfEslTAJ
[web tests]: <https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_tests.md>
[supportedPlatforms]: <https://chromium.googlesource.com/chromium/src/+/master/third_party/blink/renderer/platform/runtime_enabled_features.json5#36> [supportedPlatforms]: <https://chromium.googlesource.com/chromium/src/+/master/third_party/blink/renderer/platform/runtime_enabled_features.json5#36>
[cssProperties]: <https://chromium.googlesource.com/chromium/src/+/master/third_party/blink/renderer/core/css/css_properties.json5> [cssProperties]: <https://chromium.googlesource.com/chromium/src/+/master/third_party/blink/renderer/core/css/css_properties.json5>
[virtual test suite]: <https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_tests.md#testing-runtime-flags> [virtual test suite]: <https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_tests.md#testing-runtime-flags>
...@@ -187,3 +188,4 @@ https://groups.google.com/a/chromium.org/d/msg/blink-dev/JBakhu5J6Qs/re2LkfEslTA ...@@ -187,3 +188,4 @@ https://groups.google.com/a/chromium.org/d/msg/blink-dev/JBakhu5J6Qs/re2LkfEslTA
[runtime_enabled_features.json5]: <https://chromium.googlesource.com/chromium/src/+/master/third_party/blink/renderer/platform/runtime_enabled_features.json5> [runtime_enabled_features.json5]: <https://chromium.googlesource.com/chromium/src/+/master/third_party/blink/renderer/platform/runtime_enabled_features.json5>
[make_internal_runtime_flags.py]: <https://chromium.googlesource.com/chromium/src/+/master/third_party/blink/renderer/build/scripts/make_internal_runtime_flags.py> [make_internal_runtime_flags.py]: <https://chromium.googlesource.com/chromium/src/+/master/third_party/blink/renderer/build/scripts/make_internal_runtime_flags.py>
[code_generator_v8.py]: <https://chromium.googlesource.com/chromium/src/+/master/third_party/blink/renderer/bindings/scripts/code_generator_v8.py> [code_generator_v8.py]: <https://chromium.googlesource.com/chromium/src/+/master/third_party/blink/renderer/bindings/scripts/code_generator_v8.py>
[virtual/stable]: <https://source.chromium.org/chromium/chromium/src/+/master:third_party/blink/web_tests/VirtualTestSuites;drc=9878f26d52d32871ed1c085444196e5453909eec;l=112>
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment