Commit 24e7ae97 authored by Caleb Rouleau's avatar Caleb Rouleau Committed by Commit Bot

[Speed docs] Add media harness documentation.

R=johnchen@chromium.org, kxloh@chromium.org

Change-Id: I9b50043385af5fb93f19a6d1c4c7244754486591
bug: 853749, 949246
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1765443Reviewed-by: default avatarJohn Chen <johnchen@chromium.org>
Commit-Queue: Caleb Rouleau <crouleau@chromium.org>
Auto-Submit: Caleb Rouleau <crouleau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#690025}
parent b62ef862
# Media Benchmarks
There is no detailed external version of the documentation for media Telemetry harness.
Googlers can find the internal documentation here: http://go/videostack-chromeperf-sheriff
## Overview
The media benchmarking harness consists of the
[media story set](../../../../tools/perf/page_sets/media_cases.py),
and the [media
metric](https://chromium.googlesource.com/catapult.git/+/HEAD/tracing/tracing/metrics/media_metric.html).
Those two elements are brought together by the
[media benchmark](../../../../tools/perf/benchmarks/media.py).
The media story set is a set webpages along with
instructions for automating interactions with them. The media story set consists
of two types of stories: src= stories and MSE stories. The src= stories simply
run a saved media file from start to end. The MSE stories directly call various
MSE APIs.
The media metric takes in a trace file and computes various metrics such as
seek time and time to play using the trace events.
...@@ -28,8 +28,8 @@ kraken,hablich@chromium.org,Blink>JavaScript,, ...@@ -28,8 +28,8 @@ kraken,hablich@chromium.org,Blink>JavaScript,,
load_library_perf_tests,"xhwang@chromium.org, crouleau@chromium.org",Internals>Media>Encrypted,, load_library_perf_tests,"xhwang@chromium.org, crouleau@chromium.org",Internals>Media>Encrypted,,
loading.desktop,"kouhei@chromium.org, ksakamoto@chromium.org",Blink>Loader,https://bit.ly/loading-benchmarks,"cache_temperature_cold,cache_temperature_warm,international,intl_ar_fa_he,intl_es_fr_pt_BR,intl_hi_ru,intl_ja_zh,intl_ko_th_vi,typical" loading.desktop,"kouhei@chromium.org, ksakamoto@chromium.org",Blink>Loader,https://bit.ly/loading-benchmarks,"cache_temperature_cold,cache_temperature_warm,international,intl_ar_fa_he,intl_es_fr_pt_BR,intl_hi_ru,intl_ja_zh,intl_ko_th_vi,typical"
loading.mobile,"kouhei@chromium.org, ksakamoto@chromium.org",Blink>Loader,https://bit.ly/loading-benchmarks,"cache_temperature_cold,cache_temperature_hot,cache_temperature_warm,easy_ttfmp,easy_tti,global,pwa,tough_ttfmp,tough_tti" loading.mobile,"kouhei@chromium.org, ksakamoto@chromium.org",Blink>Loader,https://bit.ly/loading-benchmarks,"cache_temperature_cold,cache_temperature_hot,cache_temperature_warm,easy_ttfmp,easy_tti,global,pwa,tough_ttfmp,tough_tti"
media.desktop,dalecurtis@chromium.org,Internals>Media,,"aac,audio_only,audio_video,av1,background,beginning_to_end,busyjs,cns,h264,is_4k,is_50fps,mp3,mse,opus,seek,src,video_only,vorbis,vp8,vp9" media.desktop,dalecurtis@chromium.org,Internals>Media,https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark/harnesses/media.md,"aac,audio_only,audio_video,av1,background,beginning_to_end,busyjs,cns,h264,is_4k,is_50fps,mp3,mse,opus,seek,src,video_only,vorbis,vp8,vp9"
media.mobile,dalecurtis@chromium.org,Internals>Media,,"aac,audio_only,audio_video,background,beginning_to_end,busyjs,cns,h264,mp3,mse,opus,seek,src,video_only,vorbis,vp9" media.mobile,dalecurtis@chromium.org,Internals>Media,https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark/harnesses/media.md,"aac,audio_only,audio_video,background,beginning_to_end,busyjs,cns,h264,mp3,mse,opus,seek,src,video_only,vorbis,vp9"
media_perftests,"crouleau@chromium.org, dalecurtis@chromium.org",Internals>Media,, media_perftests,"crouleau@chromium.org, dalecurtis@chromium.org",Internals>Media,,
memory.desktop,erikchen@chromium.org,,, memory.desktop,erikchen@chromium.org,,,
net_perftests,net-dev@chromium.org,Internals>Network,, net_perftests,net-dev@chromium.org,Internals>Network,,
......
...@@ -45,7 +45,8 @@ class _MediaBenchmark(perf_benchmark.PerfBenchmark): ...@@ -45,7 +45,8 @@ class _MediaBenchmark(perf_benchmark.PerfBenchmark):
@benchmark.Info(emails=['dalecurtis@chromium.org'], @benchmark.Info(emails=['dalecurtis@chromium.org'],
component='Internals>Media') component='Internals>Media',
documentation_url='https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark/harnesses/media.md') # pylint: disable=line-too-long
class MediaDesktop(_MediaBenchmark): class MediaDesktop(_MediaBenchmark):
"""Obtains media performance for key user scenarios on desktop.""" """Obtains media performance for key user scenarios on desktop."""
SUPPORTED_PLATFORMS = [story.expectations.ALL_DESKTOP] SUPPORTED_PLATFORMS = [story.expectations.ALL_DESKTOP]
...@@ -60,7 +61,8 @@ class MediaDesktop(_MediaBenchmark): ...@@ -60,7 +61,8 @@ class MediaDesktop(_MediaBenchmark):
# If any story is failing on svelte, please only disable on svelte. # If any story is failing on svelte, please only disable on svelte.
@benchmark.Info(emails=['dalecurtis@chromium.org'], @benchmark.Info(emails=['dalecurtis@chromium.org'],
component='Internals>Media') component='Internals>Media',
documentation_url='https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark/harnesses/media.md') # pylint: disable=line-too-long
class MediaMobile(_MediaBenchmark): class MediaMobile(_MediaBenchmark):
"""Obtains media performance for key user scenarios on mobile devices.""" """Obtains media performance for key user scenarios on mobile devices."""
......
...@@ -15,8 +15,6 @@ UNDOCUMENTED_BENCHMARKS = { ...@@ -15,8 +15,6 @@ UNDOCUMENTED_BENCHMARKS = {
'jetstream', 'jetstream',
'kraken', 'kraken',
'load_library_perf_tests', 'load_library_perf_tests',
'media.desktop',
'media.mobile',
'media_perftests', 'media_perftests',
'memory.desktop', 'memory.desktop',
'net_perftests', 'net_perftests',
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment