Commit 8b4f63d8 authored by zhaoyangli's avatar zhaoyangli Committed by Commit Bot

[iOS][test runner] Handle "* encountered an error" failure in Xcode log

Since some beta version of Xcode 12, XCTest result can report XCTest
runner app crash as a failed test in log. In this CL, log parser reports
this failure as "BUILD_INTERRUPTED" which fits into the existing logic
to handle test process crash in test runner scripts.

Bug: 1115232
Change-Id: Icebdb5456005b2473e5ed8af683487ac532cda3b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2350442Reviewed-by: default avatarJustin Cohen <justincohen@chromium.org>
Commit-Queue: Zhaoyang Li <zhaoyangli@chromium.org>
Cr-Commit-Position: refs/heads/master@{#797470}
parent 416d914f
...@@ -15,6 +15,11 @@ import subprocess ...@@ -15,6 +15,11 @@ import subprocess
import test_runner import test_runner
# Some system errors are reported as failed tests in Xcode test result log in
# Xcode 12, e.g. test app crash in xctest parallel testing. This are reported
# as 'BUILD_INTERRUPTED' in failed test log of the attempt and will be removed
# if all tests pass in re-attempts.
SYSTEM_ERROR_TEST_NAME_SUFFIXES = ['encountered an error']
LOGGER = logging.getLogger(__name__) LOGGER = logging.getLogger(__name__)
...@@ -49,7 +54,7 @@ def format_test_case(test_case): ...@@ -49,7 +54,7 @@ def format_test_case(test_case):
`[TestClass/TestMethod]` `[TestClass/TestMethod]`
Returns: Returns:
Test case id in format TestClass_TestMethod. Test case id in format TestClass/TestMethod.
""" """
return test_case.replace('[', '').replace(']', '').replace( return test_case.replace('[', '').replace(']', '').replace(
'-', '').replace(' ', '/') '-', '').replace(' ', '/')
...@@ -175,6 +180,10 @@ class Xcode11LogParser(object): ...@@ -175,6 +180,10 @@ class Xcode11LogParser(object):
continue continue
for test in test_suite['subtests']['_values']: for test in test_suite['subtests']['_values']:
test_name = test['identifier']['_value'] test_name = test['identifier']['_value']
if any(
test_name.endswith(suffix)
for suffix in SYSTEM_ERROR_TEST_NAME_SUFFIXES):
test_name = 'BUILD_INTERRUPTED'
if test['testStatus']['_value'] == 'Success': if test['testStatus']['_value'] == 'Success':
results['passed'].append(test_name) results['passed'].append(test_name)
else: else:
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment