Skip to content

GitLab

T
tangled
Commit e373ad9f authored by hongchan's avatar hongchan Committed by Commit bot
Browse files
Options

Additional unit test for audit.js: failure cases and logging examples. 

The unit test for audit.js only covers basic/successful cases.
This CL is to add another unit test that tests failure cases with the
expected text result.

In the course of refining the failure output message, some editorial
edits also have been made in audit.js.

BUG=672988
TEST=LayoutTests/webaudio/unit-tests/audit-failures.html

Review-Url: https://codereview.chromium.org/2568573002
Cr-Commit-Position: refs/heads/master@{#439681}
parent 3778159f
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 254 additions and 56 deletions
third_party/WebKit/LayoutTests/webaudio/unit-tests/audit-expected.txt 0 → 100644
View file @ e373ad9f
CONSOLE ERROR: line 39: [audit.js] this test requires the explicit comparison with the expected result when it runs with run-webkit-tests.
This is a testharness.js-based test.
PASS # AUDIT TASK RUNNER STARTED.
PASS > [numerical] Numerical assertion unit test.
PASS 2.3 is 2 within an error of 0.3.
PASS [1,1,1] contains only the constant 1.
PASS [1,1,1] is identical to the array [1,1,1].
PASS [1,1,1,1,2,2,3,3,3] contains all the expected values in the correct order: one, two, three.
PASS [0.5,0.5,0.55,0.5,0.45,0.5] has no glitch above the threshold of 0.06.
PASS < [numerical] All assertions passed. (total 5 assertions)
PASS > [basic] Simple unit tests for basic assertions.
PASS OfflineAudioContext does exist.
PASS Setting foo to 0 did not throw an exception.
PASS function () { var foo = bar; } threw ReferenceError : "bar is not defined".
PASS 3 < 5 is true.
PASS false is false.
PASS 1 is equal to 1.
PASS 1 is not equal to 2.
PASS object is equal to object.
PASS 2 is greater than 1.
PASS 2 is greater than or equal to 2.
PASS 1 is less than 2.
PASS 1 is less than or equal to 1.
PASS Decoding audio data with no argument rejected correctly with TypeError: Failed to execute 'decodeAudioData' on 'BaseAudioContext': 1 argument required, but only 0 present..
PASS Start OAC rendering resolved correctly.
PASS < [basic] All assertions passed. (total 14 assertions)
PASS # AUDIT TASK RUNNER FINISHED: 2 tasks ran successfully.
Harness: the test ran to completion.
third_party/WebKit/LayoutTests/webaudio/unit-tests/audit-failures-expected.txt 0 → 100644
View file @ e373ad9f
CONSOLE ERROR: line 39: [audit.js] this test requires the explicit comparison with the expected result when it runs with run-webkit-tests.
This is a testharness.js-based test.
PASS # AUDIT TASK RUNNER STARTED.
PASS > [basic-failure] Testing basic assertion failures.
FAIL X Setting foo to bar incorrectly threw ReferenceError: "bar is not defined". assert_true: expected true got false
FAIL X function () { var foo = 0; } did not throw an exception. assert_true: expected true got false
FAIL X 3 < 5 is not true. assert_true: expected true got false
FAIL X true is not false. assert_true: expected true got false
FAIL X 1 is not equal to 2. assert_true: expected true got false
FAIL X 1 should not be equal to 1. assert_true: expected true got false
FAIL X typeof 3.141592 is not equal to string. assert_true: expected true got false
FAIL X 1 is not greater than 2. assert_true: expected true got false
FAIL X 1 is not greater than or equal to 2. assert_true: expected true got false
FAIL X 2 is not less than 1. assert_true: expected true got false
FAIL X 2 is not less than or equal to 1. assert_true: expected true got false
FAIL X Decoding audio data with no argument rejected incorrectly with TypeError: Failed to execute 'decodeAudioData' on 'BaseAudioContext': 1 argument required, but only 0 present.. assert_true: expected true got false
FAIL X undefined resolved incorrectly. assert_true: expected true got false
FAIL < [basic-failure] 13 out of 13 assertions were failed. assert_true: expected true got false
PASS > [numerical-failures] Testing numerical assertion failures.
FAIL X 0 is not close to 0.1 within an error of 0 assert_true: expected true got false
FAIL X Expected 5 for all values but found 7 unexpected values. :
Index Actual
[0] 1
[1] 8
[2] 11
[3] 18
...and 3 more errors. assert_true: expected true got false
FAIL X [1,8,11,18,6,5,5,5,123,49] expected to be equal to the array [1,8,10,18,6,5,6,5,123,48] but differs in 3 places:
Index Actual Expected
[2] 1.1000000000000000e+1 1.0000000000000000e+1
[6] 5.0000000000000000e+0 6.0000000000000000e+0
[9] 4.9000000000000000e+1 4.8000000000000000e+1 assert_true: expected true got false
FAIL X [1,1,1,1,2,2,3,3,3] expected to have the value sequence of [1,3,2] but got 2 at index 4. assert_true: expected true got false
FAIL X [0.5,0.5,0.55,0.5,0.45,0.5] has a glitch at index 5 of size 0.04999999999999999. assert_true: expected true got false
FAIL X [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.9] does not equal [0.7,0.6,0.5,0.4,0.3,0.2,0.1] with an element-wise tolerance of {"absoluteThreshold":0.02,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[0] 1.0000000000000001e-1 6.9999999999999996e-1 5.9999999999999998e-1 8.5714285714285721e-1 2.0000000000000000e-2
[1] 2.0000000000000001e-1 5.9999999999999998e-1 3.9999999999999997e-1 6.6666666666666663e-1 2.0000000000000000e-2
[2] 2.9999999999999999e-1 5.0000000000000000e-1 2.0000000000000001e-1 4.0000000000000002e-1 2.0000000000000000e-2
[4] 5.0000000000000000e-1 2.9999999999999999e-1 2.0000000000000001e-1 6.6666666666666674e-1 2.0000000000000000e-2
[5] 5.9999999999999998e-1 2.0000000000000001e-1 3.9999999999999997e-1 1.9999999999999998e+0 2.0000000000000000e-2
Max AbsError of 5.9999999999999998e-1 at index of 0.
Max RelError of 5.9999999999999991e+0 at index of 6. assert_true: expected true got false
FAIL X [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.9] does not equal [0.7,0.6,0.5,0.4,0.3,0.2,0.1] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0.1}.
Index Actual Expected AbsError RelError Test threshold
[0] 1.0000000000000001e-1 6.9999999999999996e-1 5.9999999999999998e-1 8.5714285714285721e-1 6.9999999999999993e-2
[1] 2.0000000000000001e-1 5.9999999999999998e-1 3.9999999999999997e-1 6.6666666666666663e-1 5.9999999999999998e-2
[2] 2.9999999999999999e-1 5.0000000000000000e-1 2.0000000000000001e-1 4.0000000000000002e-1 5.0000000000000003e-2
[4] 5.0000000000000000e-1 2.9999999999999999e-1 2.0000000000000001e-1 6.6666666666666674e-1 2.9999999999999999e-2
[5] 5.9999999999999998e-1 2.0000000000000001e-1 3.9999999999999997e-1 1.9999999999999998e+0 2.0000000000000004e-2
Max AbsError of 5.9999999999999998e-1 at index of 0.
Max RelError of 5.9999999999999991e+0 at index of 6. assert_true: expected true got false
FAIL X [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.9] does not equal [0.7,0.6,0.5,0.4,0.3,0.2,0.1] with an element-wise tolerance of {"absoluteThreshold":0.02,"relativeThreshold":0.1}.
Index Actual Expected AbsError RelError Test threshold
[0] 1.0000000000000001e-1 6.9999999999999996e-1 5.9999999999999998e-1 8.5714285714285721e-1 6.9999999999999993e-2
[1] 2.0000000000000001e-1 5.9999999999999998e-1 3.9999999999999997e-1 6.6666666666666663e-1 5.9999999999999998e-2
[2] 2.9999999999999999e-1 5.0000000000000000e-1 2.0000000000000001e-1 4.0000000000000002e-1 5.0000000000000003e-2
[4] 5.0000000000000000e-1 2.9999999999999999e-1 2.0000000000000001e-1 6.6666666666666674e-1 2.9999999999999999e-2
[5] 5.9999999999999998e-1 2.0000000000000001e-1 3.9999999999999997e-1 1.9999999999999998e+0 2.0000000000000004e-2
Max AbsError of 5.9999999999999998e-1 at index of 0.
Max RelError of 5.9999999999999991e+0 at index of 6. assert_true: expected true got false
FAIL < [numerical-failures] 8 out of 8 assertions were failed. assert_true: expected true got false
FAIL # AUDIT TASK RUNNER FINISHED: 2 out of 2 tasks were failed. assert_true: expected true got false
Harness: the test ran to completion.
third_party/WebKit/LayoutTests/webaudio/unit-tests/audit-failures.html 0 → 100644
View file @ e373ad9f
<!DOCTYPE html>
<html>
<head>
<title>audit.js: handling failures</title>
<script src="../../resources/testharness.js"></script>
<script src="../../resources/testharnessreport.js"></script>
<script src="../resources/audit.js"></script>
</head>
<body>
<script>
// The task runner will print out a console message to ensure the text file
// comparison.
var audit = Audit.createTaskRunner({ requireResultFile: true });
// These assertions are supposed to be failed. The WPT (testharness) fails
// because the assertions are failing, but the existence of expected test
// result makes this test pass as long as the actual and the expected match.
// See the expected result file to see the logging message from Audit.
// Basic assertions.
audit.define('basic-failure', function (task, should) {
task.describe('Testing basic assertion failures.');
should(function () { var foo = bar; }, 'Setting foo to bar').notThrow();
should(function () { var foo = 0; }).throw('ReferenceError');
should(3 > 5, '3 < 5').beTrue();
should(true).beFalse();
should(1).beEqualTo(2)
should(1).notBeEqualTo(1)
should(typeof 3.141592, 'typeof 3.141592').beEqualTo('string');
should(1).beGreaterThan(2);
should(1).beGreaterThanOrEqualTo(2);
should(2).beLessThan(1);
should(2).beLessThanOrEqualTo(1);
let oac = new OfflineAudioContext(1, 128, 44100);
Promise.all([
should(oac.decodeAudioData(), 'Decoding audio data with no argument')
.beResolved(),
should(oac.startRendering()).beRejected()
]).then(task.done.bind(task));
});
// Numerical assertions.
audit.define('numerical-failures', function (task, should) {
task.describe('Testing numerical assertion failures.');
should(0).beCloseTo(0.1, { threshold: 0 });
should([1, 8, 11, 18, 6, 5, 5, 5, 123, 49]).beConstantValueOf(5);
should([1, 8, 11, 18, 6, 5, 5, 5, 123, 49])
.beEqualToArray([1, 8, 10, 18, 6, 5, 6, 5, 123, 48]);
should([1, 1, 1, 1, 2, 2, 3, 3, 3]).containValues([1, 3, 2]);
should([0.5, 0.5, 0.55, 0.5, 0.45, 0.5]).notGlitch(0.04);
should([0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.9])
.beCloseToArray([0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1], {
absoluteThreshold: 0.02
});
should([0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.9])
.beCloseToArray([0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1], {
relativeThreshold: 0.1
});
should([0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.9])
.beCloseToArray([0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1], {
absoluteThreshold: 0.02,
relativeThreshold: 0.1
});
task.done();
});
// With no argument, this executes all tasks in the order defined.
audit.run();
</script>
</body>
</html>
third_party/WebKit/LayoutTests/webaudio/unit-tests/audit.html
View file @ e373ad9f
...@@ -8,7 +8,8 @@ ...@@ -8,7 +8,8 @@
</head> </head>
<body> <body>
<script> <script>
var audit = Audit.createTaskRunner(); var audit = Audit.createTaskRunner({ requireResultFile: true });
// Basic assertion testing. // Basic assertion testing.
audit.define('basic', function (task, should) { audit.define('basic', function (task, should) {
...@@ -30,7 +31,8 @@ ...@@ -30,7 +31,8 @@
let oac = new OfflineAudioContext(1, 128, 44100); let oac = new OfflineAudioContext(1, 128, 44100);
Promise.all([ Promise.all([
should(oac.startRendering(), 'Start OAC rendering').beResolved(), should(oac.startRendering(), 'Start OAC rendering').beResolved(),
should(oac.decodeAudioData()).beRejected() should(oac.decodeAudioData(), 'Decoding audio data with no argument')
.beRejected()
]).then(task.done.bind(task)); ]).then(task.done.bind(task));
}); });
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment