• Robert Ma's avatar
    Make rebaseline-cl support run_wpt_tests.py · 0111f103
    Robert Ma authored
    Breakdown of this big change:
    1. Add two WPT builders to builders.json, and a new method to
       builder_list to check whether a builder runs wptrunner, which is
       needed to create special cases for WPT (see below).
    2. Relax conditions in results_fetcher to include wpt_tests_suite, too.
    3. Add a new helper method get_file_path_for_wpt_test to base.py with
       some cleanup to make param names more consistent.
    4. Add a "non-prefix mode" to TestBaselineSet, where added tests are
       treated as the full test paths without lookup via the port object.
       This is needed because test paths on WPT builders are not relative to
       web_tests (but instead to web_tests/external/wpt); it also makes
       sense as rebaseline-cl always adds specific tests instead of test
       prefixes.
    5. Two special cases for WPT builders in rebaseline.py that will stick:
       a. _file_name_for_expected_result looks up the real test file of a
          WPT test (to work with variants, etc.) and add ".ini" instead of
          "-expected.txt" to the filename.
       b. _suffixes_for_actual_failures unconditionally returns "txt" for a
          WPT test, ignoring other artifacts (e.g. png for reftests).
    6. Three special cases for WPT builders that need to be reverted when
       we have full-platform coverage (marked with TODO), all related to the
       optimizer:
       a: Skip copy-existing-baselines-internal in rebaseline.py.
       b: Skip _optimize_baselines in rebaseline.py.
       c. Directly download to the generic baseline path instead of the
          platform-specific one in rebaseline_test.py.
    
    Drive-by:
    1. To make test setup easier, _add_manifest_to_mock_file_system is moved
       from base.py to test.py as a new public helper.
    2. Add or tweak the __repr__ methods of some classes for clearer debug
       logging.
    3. Reformat using "git cl format".
    
    Bug: 1154085
    Change-Id: I4c4434f73f68d9848267579f1ef544a6b05903cd
    Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2575859
    Commit-Queue: Robert Ma <robertma@chromium.org>
    Reviewed-by: default avatarLuke Z <lpz@chromium.org>
    Cr-Commit-Position: refs/heads/master@{#835872}
    0111f103
rebaseline.py 27.1 KB