Skip to content

GitLab

T
tangled
Commit 75b64868 authored by Douglas Creager's avatar Douglas Creager Committed by Commit Bot
Browse files
Options

Network Error Logging: Add web platform tests 

This adds web-platform-tests for Network Error Logging.  They should be
suitable to upstream into the cross-platform WPT repo.

Bug: 748549
Change-Id: I24098aeec8488c34a5f9239309264e9534dca6e4
Reviewed-on: https://chromium-review.googlesource.com/1106518Reviewed-by: default avatarPeter Beverloo <peter@chromium.org>
Reviewed-by: default avatarPhilip Jägenstedt <foolip@chromium.org>
Commit-Queue: Douglas Creager <dcreager@chromium.org>
Cr-Commit-Position: refs/heads/master@{#584093}
parent dbce3023
Hide whitespace changes
Inline Side-by-side
Showing with 646 additions and 0 deletions
content/shell/browser/shell_url_request_context_getter.cc
View file @ 75b64868
......@@ -197,6 +197,9 @@ net::URLRequestContext* ShellURLRequestContextGetter::GetURLRequestContext() {
base::TimeDelta::FromMilliseconds(100);
builder.set_reporting_policy(std::move(reporting_policy));
}
builder.set_network_error_logging_enabled(
base::FeatureList::IsEnabled(network::features::kNetworkErrorLogging));
#endif // BUILDFLAG(ENABLE_REPORTING)
builder.set_enable_brotli(true);
......
third_party/WebKit/LayoutTests/TestExpectations
View file @ 75b64868
......@@ -2613,6 +2613,9 @@ crbug.com/704259 external/wpt/content-security-policy/reporting/reporting-api-re
crbug.com/704259 external/wpt/content-security-policy/reporting/reporting-api-report-to-overrides-report-uri-2.https.sub.html [ Skip ]
crbug.com/704259 external/wpt/content-security-policy/reporting/reporting-api-sends-reports-on-violation.https.sub.html [ Skip ]
# Remove these from virtual tests when Network Error Logging is turned on by default
crbug.com/748549 external/wpt/network-error-logging [ Skip ]
crbug.com/863896 http/tests/permissions/test-query.html [ Timeout ]
# This test requires enabling the flag CustomUserTiming in a virtual test.
......
third_party/WebKit/LayoutTests/VirtualTestSuites
View file @ 75b64868
......@@ -596,6 +596,11 @@
"base": "external/wpt/content-security-policy/reporting-api",
"args": ["--enable-features=Reporting"]
},
{
"prefix": "network-error-logging",
"base": "external/wpt/network-error-logging",
"args": ["--enable-features=Reporting,NetworkErrorLogging"]
},
{
"prefix": "mouseevent_fractional",
"base": "fast/events",
......
third_party/WebKit/LayoutTests/external/wpt/network-error-logging/README.md 0 → 100644
View file @ 75b64868
# Network Error Logging
The tests in this directory exercise the user agent's implementation of [Network
Error Logging](https://w3c.github.io/network-error-logging/) and
[Reporting](https://w3c.github.io/reporting/).
## Collector
Each test case generates a unique `reportID` that is used to distinguish the NEL
reports generated by that test case.
The [support/report.py][] file is a [Python file handler][] that can be used as
a Reporting collector. Its default operation is to save any reports that it
receives into the [stash][]. If you pass in the optional `op` URL parameter,
with a value of `retrieve_report`, it will instead return a list of all of the
reports received for a particular `reportID`.
[Python file handler]: https://wptserve.readthedocs.io/en/latest/handlers.html#python-file-handlers
[stash]: https://wptserve.readthedocs.io/en/latest/stash.html
[support/report.py]: support/report.py
## Installing NEL policies
NEL reports are only generated if the user agent has received a NEL policy for
the origin of the request. The current request counts; if its response contains
a policy, that policy is used for the current request and all future requests,
modulo the policy's `max_age` field.
Most of the test cases will therefore make a request or two to install NEL
policies, and then make another request that should or should not be covered by
those policies. It will then assert that NEL reports were or were not created,
as required by the spec.
The [support][] directory contains several images, each of which defines a
particular "kind" of NEL policy (e.g., `include_subdomains` set vs unset, no
policy at all, etc.). The [support/nel.sub.js][] file contains helper
JavaScript methods for requesting those images, so that the test cases
themselves are more descriptive.
[support]: support
[support/nel.sub.js]: support/nel.sub.js
## Avoiding spurious reports
NEL policies apply to **all** future requests to the origin. We therefore serve
all of the test case's "infrastructure" (the test case itself,
[support/report.py][] and [support/nel.sub.js][]) on a different origin than
the requests that exercise the NEL implementation. That ensures that we don't
have to wade through NEL reports about the infrastructure when verifying the NEL
reports about the requests that we care about.
## Browser configuration
You must configure your browser's Reporting implementation to upload reports for
a request immediately. The test cases do not currently have any timeouts; they
assume that as soon as the Fetch API promise resolves, any NEL reports for the
request have already been uploaded.
## Test parallelism
Because NEL policies are stored in a global cache in the user agent, we need to
run the tests in this directory serially instead of in parallel. We implement a
simple spin-lock in [support/lock.py][] to ensure that only one test is allowed
to perform any NEL-related requests at a time.
[support/lock.py]: support/lock.py
## CORS preflights
Reporting uploads are subject to CORS preflights. We want to test normal
operation (when preflight requests succeed) as well as failures of the CORS
preflight logic in the user agent. To support this, our test collector is
configured to always reject the CORS preflight for a single domain (www2), and
to always grant the CORS preflight for all other test subdomains.
third_party/WebKit/LayoutTests/external/wpt/network-error-logging/no-report-on-failed-cors-preflight.https.html 0 → 100644
View file @ 75b64868
<!DOCTYPE HTML>
<html>
<head>
<title>
Test that NEL reports are not sent if the CORS preflight fails
</title>
<script src='/resources/testharness.js'></script>
<script src='/resources/testharnessreport.js'></script>
<script src='./support/nel.sub.js'></script>
</head>
<body>
<script>
nel_test(async t => {
// Make a request to a resource whose response headers include a NEL
// policy, but where the report uploader will reject the CORS preflight.
await fetchResourceWithBasicPolicy('www2');
// Because the CORS preflight is rejected, we should never receive a
// report about the request.
assert_false(await reportExists({
url: getURLForResourceWithBasicPolicy('www2'),
type: "network-error",
}));
});
</script>
</body>
</html>
third_party/WebKit/LayoutTests/external/wpt/network-error-logging/no-report-on-subdomain-404.https.html 0 → 100644
View file @ 75b64868
<!DOCTYPE HTML>
<html>
<head>
<title>
Test that include_subdomains policies do NOT report HTTP errors
</title>
<script src='/resources/testharness.js'></script>
<script src='/resources/testharnessreport.js'></script>
<script src='./support/nel.sub.js'></script>
</head>
<body>
<script>
nel_test(async t => {
// Make a request to a resource whose response headers include an
// include_subdomains NEL policy.
await fetchResourceWithIncludeSubdomainsPolicy();
// Make a request to another resource on a subdomain of the above. This
// resource doesn't exist, so the server should return a 404.
await fetchMissingResource('www');
// The include_subdomains policy that we just received should NOT cover
// the second request, since include_subdomains policies can only report
// on DNS errors.
assert_false(await reportExists({
url: getURLForMissingResource('www'),
type: "network-error",
}));
});
</script>
</body>
</html>
third_party/WebKit/LayoutTests/external/wpt/network-error-logging/no-report-on-subdomain-success.https.html 0 → 100644
View file @ 75b64868
<!DOCTYPE HTML>
<html>
<head>
<title>
Test that include_subdomains policies do NOT report successful requests
</title>
<script src='/resources/testharness.js'></script>
<script src='/resources/testharnessreport.js'></script>
<script src='./support/nel.sub.js'></script>
</head>
<body>
<script>
nel_test(async t => {
// Make a request to a resource whose response headers include an
// include_subdomains NEL policy.
await fetchResourceWithIncludeSubdomainsPolicy();
// Make a request to another resource on a subdomain of the above, which
// does not define its own NEL policy.
await fetchResourceWithNoPolicy('www');
// The include_subdomains policy that we just received should NOT cover
// the second request, since include_subdomains policies can only report
// on DNS errors.
assert_false(await reportExists({
url: getURLForResourceWithNoPolicy('www'),
type: "network-error",
}));
});
</script>
</body>
</html>
third_party/WebKit/LayoutTests/external/wpt/network-error-logging/reports-are-not-observable.https.html 0 → 100644
View file @ 75b64868
<!DOCTYPE HTML>
<html>
<head>
<title>
Test that NEL reports are not observable from JavaScript
</title>
<script src='/resources/testharness.js'></script>
<script src='/resources/testharnessreport.js'></script>
<script src='./support/nel.sub.js'></script>
</head>
<body>
<script>
nel_test(async t => {
// Register an observer for NEL reports.
var observer = new ReportingObserver((reports, _) => {
assert_unreached("NEL reports should not be observable");
}, {"types": ["network-error"]});
observer.observe();
// Make a request to a resource whose response headers include a NEL
// policy. If NEL reports are observable, this will queue a task that
// calls the observer function above (which we don't want).
await fetchResourceWithBasicPolicy();
// Wait for one second to give any observer callback task a chance to
// fire.
await new Promise(resolve => t.step_timeout(resolve, 1000 /* msec */));
});
</script>
</body>
</html>
third_party/WebKit/LayoutTests/external/wpt/network-error-logging/sends-report-on-404.https.html 0 → 100644
View file @ 75b64868
<!DOCTYPE HTML>
<html>
<head>
<title>
Test that NEL reports are sent for HTTP errors
</title>
<script src='/resources/testharness.js'></script>
<script src='/resources/testharnessreport.js'></script>
<script src='./support/nel.sub.js'></script>
</head>
<body>
<script>
nel_test(async t => {
// Make a request to a resource whose response headers include a NEL
// policy.
await fetchResourceWithBasicPolicy();
// Make a request to another resource on the same domain. This resource
// doesn't exist, so the server should return a 404.
await fetchMissingResource();
// The 404 won't contain its own NEL policy, but the policy we received in
// the first request should cover the second request, too, since they're
// at the same origin, so the collector should have received a report
// about it.
assert_true(await reportExists({
url: getURLForMissingResource(),
user_agent: navigator.userAgent,
type: "network-error",
body: {
method: "GET",
sampling_fraction: 1.0,
status_code: 404,
phase: "application",
type: "http.error",
},
metadata: {
content_type: "application/reports+json",
},
}));
});
</script>
</body>
</html>
third_party/WebKit/LayoutTests/external/wpt/network-error-logging/sends-report-on-subdomain-dns-failure.https.html 0 → 100644
View file @ 75b64868
<!DOCTYPE HTML>
<html>
<head>
<title>
Test that include_subdomains policies report DNS failures for subdomains
</title>
<script src='/resources/testharness.js'></script>
<script src='/resources/testharnessreport.js'></script>
<script src='./support/nel.sub.js'></script>
</head>
<body>
<script>
nel_test(async t => {
// Make a request to a resource whose response headers include an
// include_subdomains NEL policy.
await fetchResourceWithIncludeSubdomainsPolicy();
// Make a request to another resource on a nonexistent subdomain of the
// above. Since the subdomain doesn't exist, the request should fail with
// a DNS error.
await fetchResourceWithNoPolicy('nonexistent').then((response) => {
assert_unreached("Request to nonexistent domain should fail");
}, (err) => {
// Silence the error, since it's expected.
});
// The include_subdomains policy that we just received should cover the
// second request, since include_subdomains policies can report on DNS
// errors, so the collector should have received a report about it.
assert_true(await reportExists({
url: getURLForResourceWithNoPolicy('nonexistent'),
user_agent: navigator.userAgent,
type: "network-error",
body: {
method: "GET",
sampling_fraction: 1.0,
status_code: 0,
phase: "dns",
type: "dns.name_not_resolved",
},
metadata: {
content_type: "application/reports+json",
},
}));
});
</script>
</body>
</html>
third_party/WebKit/LayoutTests/external/wpt/network-error-logging/sends-report-on-success-with-subdomain-policy.https.html 0 → 100644
View file @ 75b64868
<!DOCTYPE HTML>
<html>
<head>
<title>
Test that NEL reports are sent for successful requests
</title>
<script src='/resources/testharness.js'></script>
<script src='/resources/testharnessreport.js'></script>
<script src='./support/nel.sub.js'></script>
</head>
<body>
<script>
nel_test(async t => {
// Make a request to a resource whose response headers include an
// include_subdomains NEL policy.
await fetchResourceWithIncludeSubdomainsPolicy();
// That policy should apply to the request that delivered it. Even though
// the policy has include_subdomains set, it SHOULD generate a full,
// non-downgraded report about the request, since the request has the
// same origin as the policy. (I.e., since the origins are the same, the
// include_subdomains setting is irrelevant.)
assert_true(await reportExists({
url: getURLForResourceWithIncludeSubdomainsPolicy(),
user_agent: navigator.userAgent,
type: "network-error",
body: {
method: "GET",
sampling_fraction: 1.0,
status_code: 200,
phase: "application",
type: "ok",
},
metadata: {
content_type: "application/reports+json",
},
}));
});
</script>
</body>
</html>
third_party/WebKit/LayoutTests/external/wpt/network-error-logging/sends-report-on-success.https.html 0 → 100644
View file @ 75b64868
<!DOCTYPE HTML>
<html>
<head>
<title>
Test that NEL reports are sent for successful requests
</title>
<script src='/resources/testharness.js'></script>
<script src='/resources/testharnessreport.js'></script>
<script src='./support/nel.sub.js'></script>
</head>
<body>
<script>
nel_test(async t => {
// Make a request to a resource whose response headers include a NEL
// policy.
await fetchResourceWithBasicPolicy();
// That policy should apply to the request that delivered it, so the
// collector should have received a report about the request.
assert_true(await reportExists({
url: getURLForResourceWithBasicPolicy(),
user_agent: navigator.userAgent,
type: "network-error",
body: {
method: "GET",
sampling_fraction: 1.0,
status_code: 200,
phase: "application",
type: "ok",
},
metadata: {
content_type: "application/reports+json",
},
}));
});
</script>
</body>
</html>
third_party/WebKit/LayoutTests/external/wpt/network-error-logging/support/clear-policy-pass.png.sub.headers 0 → 100644
View file @ 75b64868
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Cache-Control: post-check=0, pre-check=0, false
Pragma: no-cache
Report-To: { "group": "nel-group", "max_age": 0, "endpoints": [] }
NEL: {"max_age": 0}
third_party/WebKit/LayoutTests/external/wpt/network-error-logging/support/lock.py 0 → 100644
View file @ 75b64868
_LOCK_KEY = "network-error-logging:lock"
_TIMEOUT = 5 # seconds
def wait_for_lock(request):
t0 = time.time()
while time.time() - t0 < _TIMEOUT:
time.sleep(0.5)
value = request.server.stash.take(key=_LOCK_KEY)
if value is None:
return True
return False
def lock(request, report_id):
with request.server.stash.lock:
# Loop until the lock is free
if not wait_for_lock(request):
return (503, [], "Cannot obtain lock")
request.server.stash.put(key=_LOCK_KEY, value=report_id)
return "Obtained lock for %s" % report_id
def unlock(request, report_id):
with request.server.stash.lock:
lock_holder = request.server.stash.take(key=_LOCK_KEY)
if lock_holder != request_id:
# Return the lock holder to the stash
request.server.stash.put(key=_LOCK_KEY, value=lock_holder)
return (503, [], "Cannot release lock held by %s" % lock_holder)
return "Released lock for %s" % report_id
def main(request, response):
op = request.GET.first("op")
report_id = request.GET.first("reportID")
if op == "lock":
return lock(request, report_id)
elif op == "unlock":
return unlock(request, report_id)
else:
return (400, [], "Invalid op")
third_party/WebKit/LayoutTests/external/wpt/network-error-logging/support/nel.sub.js 0 → 100644
View file @ 75b64868
const reportID = "{{$id:uuid()}}";
/*
* NEL tests have to run serially, since the user agent maintains a global cache
* of Reporting and NEL policies, and we don't want the policies for multiple
* tests to interfere with each other. These functions (along with a Python
* handler in lock.py) implement a simple spin lock.
*/
function obtainNELLock() {
return fetch("/network-error-logging/support/lock.py?op=lock&reportID=" + reportID);
}
function releaseNELLock() {
return fetch("/network-error-logging/support/lock.py?op=unlock&reportID=" + reportID);
}
function nel_test(callback, name, properties) {
promise_test(async t => {
await obtainNELLock();
await clearReportingAndNELConfigurations();
await callback(t);
await releaseNELLock();
}, name, properties);
}
/*
* Helper functions for constructing domain names that contain NEL policies.
*/
function _monitoredDomain(subdomain) {
if (subdomain == "www") {
return "{{hosts[alt][www]}}"
} else if (subdomain == "www1") {
return "{{hosts[alt][www1]}}"
} else if (subdomain == "www2") {
return "{{hosts[alt][www2]}}"
} else if (subdomain == "nonexistent") {
return "{{hosts[alt][nonexistent]}}"
} else {
return "{{hosts[alt][]}}"
}
}
function _getNELResourceURL(subdomain, suffix) {
return "https://" + _monitoredDomain(subdomain) +
":{{ports[https][0]}}/network-error-logging/support/" + suffix;
}
/*
* Fetches a resource whose headers define a basic NEL policy (i.e., with no
* include_subdomains flag). We ensure that we request the resource from a
* different origin than is used for the main test case HTML file or for report
* uploads. This minimizes the number of reports that are generated for this
* policy.
*/
function getURLForResourceWithBasicPolicy(subdomain) {
return _getNELResourceURL(subdomain, "pass.png?id="+reportID);
}
function fetchResourceWithBasicPolicy(subdomain) {
const url = getURLForResourceWithBasicPolicy(subdomain);
return fetch(url, {mode: "no-cors"});
}
/*
* Fetches a resource whose headers define an include_subdomains NEL policy.
*/
function getURLForResourceWithIncludeSubdomainsPolicy(subdomain) {
return _getNELResourceURL(subdomain, "subdomains-pass.png?id="+reportID);
}
function fetchResourceWithIncludeSubdomainsPolicy(subdomain) {
const url = getURLForResourceWithIncludeSubdomainsPolicy(subdomain);
return fetch(url, {mode: "no-cors"});
}
/*
* Fetches a resource whose headers do NOT define a NEL policy. This may or may
* not generate a NEL report, depending on whether you've already successfully
* requested a resource from the same origin that included a NEL policy.
*/
function getURLForResourceWithNoPolicy(subdomain) {
return _getNELResourceURL(subdomain, "no-policy-pass.png");
}
function fetchResourceWithNoPolicy(subdomain) {
const url = getURLForResourceWithNoPolicy(subdomain);
return fetch(url, {mode: "no-cors"});
}
/*
* Fetches a resource that doesn't exist. This may or may not generate a NEL
* report, depending on whether you've already successfully requested a resource
* from the same origin that included a NEL policy.
*/
function getURLForMissingResource(subdomain) {
return _getNELResourceURL(subdomain, "nonexistent.png");
}
function fetchMissingResource(subdomain) {
const url = getURLForMissingResource(subdomain);
return fetch(url, {mode: "no-cors"});
}
/*
* Fetches resources that clear out any existing Reporting or NEL configurations
* for all origins that any test case might use.
*/
function getURLForClearingConfiguration(subdomain) {
return _getNELResourceURL(subdomain, "clear-pass.png?id="+reportID);
}
async function clearReportingAndNELConfigurations(subdomain) {
await Promise.all([
fetch(getURLForClearingConfiguration(""), {mode: "no-cors"}),
fetch(getURLForClearingConfiguration("www"), {mode: "no-cors"}),
fetch(getURLForClearingConfiguration("www1"), {mode: "no-cors"}),
fetch(getURLForClearingConfiguration("www2"), {mode: "no-cors"}),
]);
return;
}
/*
* Returns whether all of the fields in obj1 also exist in obj2 with the same
* values. (Put another way, returns whether obj1 and obj2 are equal, ignoring
* any extra fields in obj2.)
*/
function _isSubsetOf(obj1, obj2) {
for (const prop in obj1) {
if (typeof obj1[prop] === 'object') {
if (typeof obj2[prop] !== 'object') {
return false;
}
if (!_isSubsetOf(obj1[prop], obj2[prop])) {
return false;
}
} else if (obj1[prop] != obj2[prop]) {
return false;
}
}
return true;
}
/*
* Verifies that a report was uploaded that contains all of the fields in
* expected.
*/
async function reportExists(expected) {
var timeout =
document.querySelector("meta[name=timeout][content=long]") ? 50 : 1;
var reportLocation =
"/network-error-logging/support/report.py?op=retrieve_report&timeout=" +
timeout + "&reportID=" + reportID;
const response = await fetch(reportLocation);
const json = await response.json();
for (const report of json) {
if (_isSubsetOf(expected, report)) {
return true;
}
}
return false;
}
third_party/WebKit/LayoutTests/external/wpt/network-error-logging/support/pass.png.sub.headers 0 → 100644
View file @ 75b64868
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Cache-Control: post-check=0, pre-check=0, false
Pragma: no-cache
Report-To: { "group": "nel-group", "max_age": 10886400, "endpoints": [{ "url": "https://{{hosts[][www]}}:{{ports[https][0]}}/network-error-logging/support/report.py?op=put&reportID={{GET[id]}}" }] }
NEL: {"report_to": "nel-group", "max_age": 10886400, "success_fraction": 1.0}
third_party/WebKit/LayoutTests/external/wpt/network-error-logging/support/report.py 0 → 100644
View file @ 75b64868
import time
import json
import re
def retrieve_from_stash(request, key, timeout, default_value):
t0 = time.time()
while time.time() - t0 < timeout:
time.sleep(0.5)
value = request.server.stash.take(key=key)
if value is not None:
return json.dumps(value)
return default_value
def main(request, response):
# Handle CORS preflight requests
if request.method == 'OPTIONS':
# Always reject preflights for one subdomain
if "www2" in request.headers["Origin"]:
return (400, [], "CORS preflight rejected for www2")
return [
("Content-Type", "text/plain"),
("Access-Control-Allow-Origin", "*"),
("Access-Control-Allow-Methods", "post"),
("Access-Control-Allow-Headers", "Content-Type"),
], "CORS allowed"
op = request.GET.first("op");
key = request.GET.first("reportID")
if op == "retrieve_report":
try:
timeout = float(request.GET.first("timeout"))
except:
timeout = 0.5
return [("Content-Type", "application/json")], retrieve_from_stash(request, key, timeout, '[]')
# append new reports
new_reports = json.loads(request.body)
for report in new_reports:
report["metadata"] = {
"content_type": request.headers["Content-Type"],
}
with request.server.stash.lock:
reports = request.server.stash.take(key=key)
if reports is None:
reports = []
reports.extend(new_reports)
request.server.stash.put(key=key, value=reports)
# return acknowledgement report
return [("Content-Type", "text/plain")], "Recorded report"
third_party/WebKit/LayoutTests/external/wpt/network-error-logging/support/subdomains-pass.png.sub.headers 0 → 100644
View file @ 75b64868
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Cache-Control: post-check=0, pre-check=0, false
Pragma: no-cache
Report-To: { "group": "nel-group", "max_age": 10886400, "include_subdomains": true, "endpoints": [{ "url": "https://{{hosts[][www]}}:{{ports[https][0]}}/network-error-logging/support/report.py?op=put&reportID={{GET[id]}}" }] }
NEL: {"report_to": "nel-group", "max_age": 10886400, "include_subdomains": true, "success_fraction": 1.0}
third_party/WebKit/LayoutTests/virtual/network-error-logging/external/wpt/network-error-logging/README.txt 0 → 100644
View file @ 75b64868
# This suite runs the tests in external/wpt/network-error-logging with the flag
# --enable-feature=Reporting,NetworkErrorLogging
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment