• Michael Thiessen's avatar
    Introduce @Batch annotation for batching Instrumentation tests · 6bafbf06
    Michael Thiessen authored
    Introduces the @Batch annotation, which causes the test runner to run
    tests without a separate instrumentation run for each test (in other
    words, no Activity/process restart).
    
    Lowers the per-test overhead from around 10 seconds on my device to <5ms
    for all tests with this new annotation. (Bots from ~2s to <5ms)
    
    The BATCH.UNIT_TESTS batch is for unit tests that don't require any
    globally shared setup/teardown and don't change global state.
    
    The Batch.PER_CLASS batch is for integration tests that don't require
    a restart between each test, but do require a restart before/after the
    suite runs. It is equivalent to @Batch('unique_name').
    
    In the future, batches may be added at other layers for suites that
    agree on known starting conditions, and reset to those starting
    conditions.
    
    This change also removes some dead handling of junit3 tests so that we
    can re-interpret an array of tests as a collection of tests to run in a
    single instrumentation call.
    
    Some followup work will be required to fix reported duration, as each
    test individually is reported as taking the duration the batch of tests
    took.
    
    Bug: 989569
    Change-Id: I2fee81c33363789ca63174432a3b90b4fb472097
    Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2199706
    Commit-Queue: Michael Thiessen <mthiesse@chromium.org>
    Reviewed-by: default avatarAndrew Grieve <agrieve@chromium.org>
    Reviewed-by: default avatarMohamed Heikal <mheikal@chromium.org>
    Cr-Commit-Position: refs/heads/master@{#771630}
    6bafbf06
instrumentation.md 13.3 KB