Commit bd8cc342 authored by Ned Nguyen's avatar Ned Nguyen Committed by Commit Bot

Remove references to NEEDSMANUALREBASELINE in markdown docs and testing/scripts/common.py file

NOTRY=true  # chromedriver_py_tests flake (https://crbug.com/864205)

Bug: 621126
Change-Id: I61d2722044cffc44e5bc93b5ffc7ff9d3ab6d313
Reviewed-on: https://chromium-review.googlesource.com/1142359
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: default avatarRobert Ma <robertma@chromium.org>
Reviewed-by: default avatarDirk Pranke <dpranke@chromium.org>
Reviewed-by: default avatarJohn Chen <johnchen@chromium.org>
Cr-Commit-Position: refs/heads/master@{#576513}
parent b46b8745
...@@ -142,7 +142,6 @@ failure types. ...@@ -142,7 +142,6 @@ failure types.
| `IMAGE+TEXT` | **Layout test specific, deprecated.** The test produces image and text output, both of which fail to match what we expect. Normally you will see `FAIL` instead. | | `IMAGE+TEXT` | **Layout test specific, deprecated.** The test produces image and text output, both of which fail to match what we expect. Normally you will see `FAIL` instead. |
| `LEAK` | **Layout test specific, deprecated.** Memory leaks were detected during the test execution. | | `LEAK` | **Layout test specific, deprecated.** Memory leaks were detected during the test execution. |
| `MISSING` | **Layout test specific, deprecated.** The test completed but we could not find an expected baseline to compare against. | | `MISSING` | **Layout test specific, deprecated.** The test completed but we could not find an expected baseline to compare against. |
| `NEEDSMANUALREBASELINE` | **Layout test specific, deprecated.** The expected test result is out of date and will be ignored (as above). This result may be checked in to the TestExpectations file, but the auto-rebasline-bot will ignore these entries. This should never show up as an `actual` result. |
| `NEEDSREBASELINE` | **Layout test specific, deprecated.** The expected test result is out of date and will be ignored (as above); the auto-rebaseline-bot will look for tests of this type and automatically update them. This should never show up as an `actual` result. | | `NEEDSREBASELINE` | **Layout test specific, deprecated.** The expected test result is out of date and will be ignored (as above); the auto-rebaseline-bot will look for tests of this type and automatically update them. This should never show up as an `actual` result. |
| `REBASELINE` | **Layout test specific, deprecated.** The expected test result is out of date and will be ignored (any result other than a crash or timeout will be considered as passing). This test result should only ever show up on local test runs, not on bots (it is forbidden to check in a TestExpectations file with this expectation). This should never show up as an "actual" result. | | `REBASELINE` | **Layout test specific, deprecated.** The expected test result is out of date and will be ignored (any result other than a crash or timeout will be considered as passing). This test result should only ever show up on local test runs, not on bots (it is forbidden to check in a TestExpectations file with this expectation). This should never show up as an "actual" result. |
| `SLOW` | **Layout test specific, deprecated.** The test is expected to take longer than normal to run. This should never appear as an `actual` result, but may (incorrectly) appear in the expected fields. | | `SLOW` | **Layout test specific, deprecated.** The test is expected to take longer than normal to run. This should never appear as an `actual` result, but may (incorrectly) appear in the expected fields. |
......
...@@ -131,15 +131,6 @@ depends on its arguments. ...@@ -131,15 +131,6 @@ depends on its arguments.
assuming that there are no platform-specific results for those platforms, assuming that there are no platform-specific results for those platforms,
you can add the flag `--fill-missing`. you can add the flag `--fill-missing`.
### Rebaselining manually
1. If the tests is already listed in TestExpectations as flaky, mark the test
`NeedsManualRebaseline` and comment out the flaky line so that your patch can
land without turning the tree red. If the test is not in TestExpectations,
you can add a `[ Rebaseline ]` line to TestExpectations.
2. Run `third_party/blink/tools/blink_tool.py rebaseline-expectations`
3. Post the patch created in step 2 for review.
## Kinds of expectations files ## Kinds of expectations files
* [TestExpectations](../../third_party/WebKit/LayoutTests/TestExpectations): The * [TestExpectations](../../third_party/WebKit/LayoutTests/TestExpectations): The
...@@ -212,7 +203,7 @@ The syntax of a line is roughly: ...@@ -212,7 +203,7 @@ The syntax of a line is roughly:
[third_party/blink/tools/blinkpy/web_tests/port/base.py](../../third_party/blink/tools/blinkpy/web_tests/port/base.py) [third_party/blink/tools/blinkpy/web_tests/port/base.py](../../third_party/blink/tools/blinkpy/web_tests/port/base.py)
for the meta keywords and which modifiers they represent. for the meta keywords and which modifiers they represent.
* Expectations can be one or more of `Crash`, `Failure`, `Pass`, `Rebaseline`, * Expectations can be one or more of `Crash`, `Failure`, `Pass`, `Rebaseline`,
`Slow`, `Skip`, `Timeout`, `WontFix`, `Missing`, `NeedsManualRebaseline`. `Slow`, `Skip`, `Timeout`, `WontFix`, `Missing`.
If multiple expectations are listed, the test is considered "flaky" and any If multiple expectations are listed, the test is considered "flaky" and any
of those results will be considered as expected. of those results will be considered as expected.
......
...@@ -138,8 +138,7 @@ def parse_common_test_results(json_results, test_separator='/'): ...@@ -138,8 +138,7 @@ def parse_common_test_results(json_results, test_separator='/'):
# TODO(dpranke): crbug.com/357866 - we should simplify the handling of # TODO(dpranke): crbug.com/357866 - we should simplify the handling of
# both the return code and parsing the actual results, below. # both the return code and parsing the actual results, below.
passing_statuses = ('PASS', 'SLOW', 'NEEDSREBASELINE', passing_statuses = ('PASS', 'SLOW', 'NEEDSREBASELINE')
'NEEDSMANUALREBASELINE')
for test, result in convert_trie_to_flat_paths( for test, result in convert_trie_to_flat_paths(
json_results['tests']).iteritems(): json_results['tests']).iteritems():
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment