Commit bd8cc342 authored by Ned Nguyen's avatar Ned Nguyen Committed by Commit Bot

Remove references to NEEDSMANUALREBASELINE in markdown docs and testing/scripts/common.py file

NOTRY=true  # chromedriver_py_tests flake (https://crbug.com/864205)

Bug: 621126
Change-Id: I61d2722044cffc44e5bc93b5ffc7ff9d3ab6d313
Reviewed-on: https://chromium-review.googlesource.com/1142359
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: default avatarRobert Ma <robertma@chromium.org>
Reviewed-by: default avatarDirk Pranke <dpranke@chromium.org>
Reviewed-by: default avatarJohn Chen <johnchen@chromium.org>
Cr-Commit-Position: refs/heads/master@{#576513}
parent b46b8745
......@@ -142,7 +142,6 @@ failure types.
| `IMAGE+TEXT` | **Layout test specific, deprecated.** The test produces image and text output, both of which fail to match what we expect. Normally you will see `FAIL` instead. |
| `LEAK` | **Layout test specific, deprecated.** Memory leaks were detected during the test execution. |
| `MISSING` | **Layout test specific, deprecated.** The test completed but we could not find an expected baseline to compare against. |
| `NEEDSMANUALREBASELINE` | **Layout test specific, deprecated.** The expected test result is out of date and will be ignored (as above). This result may be checked in to the TestExpectations file, but the auto-rebasline-bot will ignore these entries. This should never show up as an `actual` result. |
| `NEEDSREBASELINE` | **Layout test specific, deprecated.** The expected test result is out of date and will be ignored (as above); the auto-rebaseline-bot will look for tests of this type and automatically update them. This should never show up as an `actual` result. |
| `REBASELINE` | **Layout test specific, deprecated.** The expected test result is out of date and will be ignored (any result other than a crash or timeout will be considered as passing). This test result should only ever show up on local test runs, not on bots (it is forbidden to check in a TestExpectations file with this expectation). This should never show up as an "actual" result. |
| `SLOW` | **Layout test specific, deprecated.** The test is expected to take longer than normal to run. This should never appear as an `actual` result, but may (incorrectly) appear in the expected fields. |
......
......@@ -131,15 +131,6 @@ depends on its arguments.
assuming that there are no platform-specific results for those platforms,
you can add the flag `--fill-missing`.
### Rebaselining manually
1. If the tests is already listed in TestExpectations as flaky, mark the test
`NeedsManualRebaseline` and comment out the flaky line so that your patch can
land without turning the tree red. If the test is not in TestExpectations,
you can add a `[ Rebaseline ]` line to TestExpectations.
2. Run `third_party/blink/tools/blink_tool.py rebaseline-expectations`
3. Post the patch created in step 2 for review.
## Kinds of expectations files
* [TestExpectations](../../third_party/WebKit/LayoutTests/TestExpectations): The
......@@ -212,7 +203,7 @@ The syntax of a line is roughly:
[third_party/blink/tools/blinkpy/web_tests/port/base.py](../../third_party/blink/tools/blinkpy/web_tests/port/base.py)
for the meta keywords and which modifiers they represent.
* Expectations can be one or more of `Crash`, `Failure`, `Pass`, `Rebaseline`,
`Slow`, `Skip`, `Timeout`, `WontFix`, `Missing`, `NeedsManualRebaseline`.
`Slow`, `Skip`, `Timeout`, `WontFix`, `Missing`.
If multiple expectations are listed, the test is considered "flaky" and any
of those results will be considered as expected.
......
......@@ -138,8 +138,7 @@ def parse_common_test_results(json_results, test_separator='/'):
# TODO(dpranke): crbug.com/357866 - we should simplify the handling of
# both the return code and parsing the actual results, below.
passing_statuses = ('PASS', 'SLOW', 'NEEDSREBASELINE',
'NEEDSMANUALREBASELINE')
passing_statuses = ('PASS', 'SLOW', 'NEEDSREBASELINE')
for test, result in convert_trie_to_flat_paths(
json_results['tests']).iteritems():
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment