Run origin-isolation WPTs without automatic WPT OI

It turns out that we automatically origin-isolate web-platform.test
domains. This makes the web platform tests under
external/wpt/origin-isolation, for the opt-in origin isolation feature
(https://crbug.com/1042415), useless, since the opt-in does nothing.

This CL introduces a new virtual test suite, which toggles a new command
line flag to disable this automatic origin isolation of WPT domains.
When running this virtual test suite, then, the WPT domains will only be
origin-isolated when they are explicitly opted in by the test files, by
setting the Origin-Isolation header.

The tests are run exclusively in this virtual test suite, by adding
their default configuration to NeverFixTests.

Fixed: 1073181
Change-Id: I9db3285f1ac3a0e5b04e8d63694649d2dbe6419e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2159536
Commit-Queue: Domenic Denicola <domenic@chromium.org>
Reviewed-by: Alex Moshchuk <alexmos@chromium.org>
Reviewed-by: Łukasz Anforowicz <lukasza@chromium.org>
Cr-Commit-Position: refs/heads/master@{#763371}

content/shell/browser/web_test/web_test_content_browser_client.cc

@@ -329,14 +329,17 @@ WebTestContentBrowserClient::GetOriginsRequiringDedicatedProcess() {
       "https://devtools.oopif.test/",
   };
 
-  // On platforms with strict Site Isolation, the also isolate WPT origins for
-  // additional OOPIF coverage.
+  // When appropriate, we isolate WPT origins for additional OOPIF coverage.
   //
-  // Don't isolate WPT origins on
-  // 1) platforms where strict Site Isolation is not the default.
-  // 2) in web tests under virtual/not-site-per-process where
+  // We don't isolate WPT origins:
+  // 1) on platforms where strict Site Isolation is not the default.
+  // 2) in web tests under virtual/not-site-per-process where the
   //    --disable-site-isolation-trials switch is used.
-  if (SiteIsolationPolicy::UseDedicatedProcessesForAllSites()) {
+  // 3) in web tests under virtual/no-auto-wpt-origin-isolation where the
+  //    --disable-auto-wpt-origin-isolation switch is used.
+  if (SiteIsolationPolicy::UseDedicatedProcessesForAllSites() &&
+      !base::CommandLine::ForCurrentProcess()->HasSwitch(
+          switches::kDisableAutoWPTOriginIsolation)) {
     // The list of hostnames below is based on
     // https://web-platform-tests.org/writing-tests/server-features.html
     const char* kWptHostnames[] = {

content/shell/common/web_test/web_test_switches.cc

@@ -35,6 +35,12 @@ const char kEnableLeakDetection[] = "enable-leak-detection";
 // Encode binary web test results (images, audio) using base64.
 const char kEncodeBinary[] = "encode-binary";
 
+// Disables the automatic origin isolation of web platform test domains.
+// We normally origin-isolate them for better test coverage, but tests of opt-in
+// origin isolation need to disable this.
+const char kDisableAutoWPTOriginIsolation[] =
+    "disable-auto-wpt-origin-isolation";
+
 // This makes us disable some web-platform runtime features so that we test
 // content_shell as if it was a stable release. It is only followed when
 // kRunWebTest is set. For the features' level, see

content/shell/common/web_test/web_test_switches.h

@@ -22,6 +22,7 @@ extern const char kEnableLeakDetection[];
 extern const char kEncodeBinary[];
 extern const char kStableReleaseMode[];
 extern const char kDisableHeadlessMode[];
+extern const char kDisableAutoWPTOriginIsolation[];
 
 #if defined(OS_WIN)
 extern const char kRegisterFontFiles[];

third_party/blink/web_tests/NeverFixTests

@@ -2027,6 +2027,14 @@ crbug.com/669083 http/tests/security/frameNavigation/xss-DENIED-top-navigation-w
 # 3) site isolation is enabled without also enabling
 #    IsolatedCodeCache (tests disabled by the test expectation below).
 http/tests/devtools/isolated-code-cache/cross-origin-test.js [ Skip ]
+# ----------------------- origin isolation ------------------------------
+# Origin isolation web platform tests are for the opt-in Origin Isolation
+# feature at https://github.com/WICG/origin-isolation. It doesn't make sense to
+# test that feature when web-platform.test domains are being forced into
+# origin-isolation mode. Note that since we also want coverage of this feature
+# with site isolation enabled, we use a different virtual test suite than
+# not-site-per-process, named no-auto-web-platform-test-origin-isolation.
+external/wpt/origin-isolation/* [ Skip ]
 # ==== Tests incompatible with the default WPT Origin Isolation end here ==^^
 
 # Tests using testRunner.useUnfortunateSynchronousResizeMode occasionally fail on Win7,

third_party/blink/web_tests/TestExpectations

@@ -215,7 +215,7 @@ crbug.com/974720 [ Mac10.13 ] virtual/text-antialias/firstline/capitalize-transf
 crbug.com/974720 [ Mac10.13 ] virtual/text-antialias/firstline/capitalize-transform.html [ Pass Crash ]
 
 # This will likely be fixed as we finish off the origin isolation architecture work
-crbug.com/1067389 external/wpt/origin-isolation/parent-no-child-yes-same.sub.https.html [ Pass Failure Crash ]
+crbug.com/1067389 virtual/no-auto-wpt-origin-isolation/external/wpt/origin-isolation/parent-no-child-yes-same.sub.https.html [ Pass Failure Crash ]
 
 # Flakily timing out or failing.
 # TODO(ccameron): Investigate this.

third_party/blink/web_tests/VirtualTestSuites

@@ -483,6 +483,11 @@
               "http/tests/webaudio/autoplay-crossorigin.html"],
     "args": ["--disable-site-isolation-trials"]
   },
+  {
+    "prefix": "no-auto-wpt-origin-isolation",
+    "bases": ["external/wpt/origin-isolation"],
+    "args": ["--disable-auto-wpt-origin-isolation"]
+  },
   {
     "prefix": "focusless-spat-nav",
     "bases": [],

third_party/blink/web_tests/virtual/no-auto-wpt-origin-isolation/README.md

+This directory contains tests that require disabling the automatic process
+isolation of web platform test origins. That isolation is usually useful for
+increased test coverage, but in some tests it interferes with the system under
+test. See https://crbug.com/1073181 for background.
+
+This is separate from the `not-site-per-process` virtual test suite, which
+disables site isolation entirely. Tests running under this virtual test suite
+benefit from _site_ isolation (e.g. isolating `not-web-platform.test` from
+`web-platform.test`). They just do not get automatic origin isolation (e.g.
+isolating `web-platform.test` from `www.web-platform.test`).