Commit 5f53d960 authored by Caleb Rouleau's avatar Caleb Rouleau Committed by Commit Bot

[media telemetry] Don't edit commandline flag defaults to exclude stories.

Remove "options = {'story_tag_filter_exclude': 'is_4k,is_50fps'}"
This isn't really a good way to do this because of the following.

if you run the code like this:
$ ./tools/perf/run_benchmark run media.mobile --browser=android-chrome
then the filter will work:
   number of stories 22

but if you run the code like this then
$ ./tools/perf/run_benchmark run media.mobile --browser=android-chrome --story-tag-filter-exclude='other_tag'
then the filter will not work:
   number of stories 29 (which means the 4k stories and 50fps stories were not excluded.)
This is because the "options" simply sets the default "--story-tag-filter-exclude"
So when a user provides a different value, the default is overriden.

My change fixes this such that you can use that flag without side effects.
It also helps by making the other parts of the system be able to know
by default what stories are in the page_set. You can see that this is
true because the the autogenerated benchmark.csv file now has fewer
tags for media.mobile because of the filtered stories.


I am attempting to redesign this
system a bit for https://goo.gl/JZ2yZQ. I noticed this code.


Also remove
tag = 'android'
since it doesn't do anything.


Bug: 894254
Change-Id: Ibf5f90e83062c785934af334c8dd6bfde89cef25
Reviewed-on: https://chromium-review.googlesource.com/c/1356320
Commit-Queue: Caleb Rouleau <crouleau@chromium.org>
Reviewed-by: default avatarJuan Antonio Navarro Pérez <perezju@chromium.org>
Cr-Commit-Position: refs/heads/master@{#612840}
parent 22d1b711
......@@ -27,7 +27,7 @@ load_library_perf_tests,"xhwang@chromium.org, crouleau@chromium.org",Internals>M
loading.desktop,"kouhei@chromium.org, ksakamoto@chromium.org",,https://bit.ly/loading-benchmarks,"cache_temperature_cold,cache_temperature_warm,international,intl_ar_fa_he,intl_es_fr_pt_BR,intl_hi_ru,intl_ja_zh,intl_ko_th_vi,typical"
loading.mobile,"kouhei@chromium.org, ksakamoto@chromium.org",,https://bit.ly/loading-benchmarks,"cache_temperature_cold,cache_temperature_hot,cache_temperature_warm,easy_ttfmp,easy_tti,global,pwa,tough_ttfmp,tough_tti"
media.desktop,dalecurtis@chromium.org,Internals>Media,,"aac,audio_only,audio_video,background,beginning_to_end,busyjs,cns,h264,is_4k,is_50fps,mp3,mse,opus,pcm,seek,src,video_only,vorbis,vp8,vp9"
media.mobile,dalecurtis@chromium.org,Internals>Media,,"aac,audio_only,audio_video,background,beginning_to_end,busyjs,cns,h264,is_4k,is_50fps,mp3,mse,opus,pcm,seek,src,video_only,vorbis,vp8,vp9"
media.mobile,dalecurtis@chromium.org,Internals>Media,,"aac,audio_only,audio_video,background,beginning_to_end,busyjs,cns,h264,mp3,mse,opus,pcm,seek,src,video_only,vorbis,vp9"
media_perftests,"crouleau@chromium.org, dalecurtis@chromium.org",Internals>Media,,
memory.desktop,erikchen@chromium.org,,,
memory.long_running_idle_gmail_background_tbmv2,ulan@chromium.org,,,
......
......@@ -28,9 +28,6 @@ class _MediaBenchmark(perf_benchmark.PerfBenchmark):
"""Base class for TBMv2-based media benchmarks (MediaDesktop and
MediaMobile)."""
def CreateStorySet(self, options):
return page_sets.MediaCasesStorySet(measure_memory=True)
def CreateCoreTimelineBasedMeasurementOptions(self):
category_filter = chrome_trace_category_filter.ChromeTraceCategoryFilter()
......@@ -68,6 +65,9 @@ class MediaDesktop(_MediaBenchmark):
"""Obtains media performance for key user scenarios on desktop."""
SUPPORTED_PLATFORMS = [story.expectations.ALL_DESKTOP]
def CreateStorySet(self, options):
return page_sets.MediaCasesDesktopStorySet(measure_memory=True)
@classmethod
def Name(cls):
return 'media.desktop'
......@@ -79,10 +79,11 @@ class MediaDesktop(_MediaBenchmark):
class MediaMobile(_MediaBenchmark):
"""Obtains media performance for key user scenarios on mobile devices."""
tag = 'android'
options = {'story_tag_filter_exclude': 'is_4k,is_50fps'}
SUPPORTED_PLATFORMS = [story.expectations.ANDROID_NOT_WEBVIEW]
def CreateStorySet(self, options):
return page_sets.MediaCasesMobileStorySet(measure_memory=True)
@classmethod
def Name(cls):
return 'media.mobile'
......
This diff is collapsed.
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment