Commit b45c3f1e authored by Matt Wolenetz's avatar Matt Wolenetz Committed by Chromium LUCI CQ

[MSE] Begin relaxing aSB() and cT() codec-specificity

Begins relaxation with a narrow case: addSourceBuffer("video/mp4") and
changeType("video/mp4") may now succeed, though any actual media
appended to them later must be encrypted HEVC content, and even then,
only on ChromeOS. Later changes will expand the relaxation of
codec-specificity further beyond this.

Specifically, this change:
1) Updates MediaSource and SourceBuffer to continue to require
   codec-specificity for MediaSource.isTypeSupported(), but includes a
   short-term special casing for addSourceBuffer and changeType to
   support just "video/mp4" on ChromeOS with HEVC EME support.

2) Updates Chromium StreamParserFactory to let parser creation occur
   with assumed eventual HEVC codec track when just "video/mp4" is used
   as addSourceBuffer() or changeType() parameter, but only on ChromeOS
   with HEVC EME support.

3) Updates Chromium SourceBufferState to require an HEVC track reported
   by the StreamParser to be disallowed with DCHECK failure if
   BUILDFLAG(ENABLE_PLATFORM_HEVC) is not true, as a protection against
   possible regression later in StreamParser function.

4) Updates Chromium SourceBufferState to require an HEVC track reported
   by the StreamParser to be encrypted (otherwise parse failure) if
   BUILDFLAG(USE_CHROMEOS_PROTECTED_MEDIA) is true.

5) Until wider relaxation is done, this change inserts a hardcoded
   hevc codec string in SourceBufferState's |expected_codecs| when just
   "video/mp4" is used for addSourceBuffer or changeType on
   ChromeOS+HEVC, enabling re-use of existing codec-specificity checks
   in SourceBufferState.

BUG=chromium:535738,b:153111783

Change-Id: I353cdbdb3c1fdafbeacacc6b7c8c39430be7d798
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2587494Reviewed-by: default avatarChrome Cunningham <chcunningham@chromium.org>
Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#837916}
parent 7600aa69
...@@ -28,6 +28,7 @@ ...@@ -28,6 +28,7 @@
#include "media/filters/frame_processor.h" #include "media/filters/frame_processor.h"
#include "media/filters/source_buffer_stream.h" #include "media/filters/source_buffer_stream.h"
#include "media/filters/stream_parser_factory.h" #include "media/filters/stream_parser_factory.h"
#include "media/media_buildflags.h"
using base::TimeDelta; using base::TimeDelta;
...@@ -52,6 +53,17 @@ std::unique_ptr<media::StreamParser> CreateParserForTypeAndCodecs( ...@@ -52,6 +53,17 @@ std::unique_ptr<media::StreamParser> CreateParserForTypeAndCodecs(
// for a few mime types that have an implicit codec. // for a few mime types that have an implicit codec.
std::string ExpectedCodecs(const std::string& content_type, std::string ExpectedCodecs(const std::string& content_type,
const std::string& codecs) { const std::string& codecs) {
#if BUILDFLAG(ENABLE_PLATFORM_HEVC) && BUILDFLAG(USE_CHROMEOS_PROTECTED_MEDIA)
// TODO(crbug.com/535738): Undo this hardcoded assumption of a valid HEVC
// codec string when codec specificity is widely relaxed. We use a valid HEVC
// string here in short-term to let the codec-specificity checks in
// SourceBufferState::OnNewConfigs require an HEVC decoder config (with not
// necessarily matching profile/level, just codec) be emitted from parsers
// initialized by addSourceBuffer or changeType with just "video/mp4".
if (codecs == "" && content_type == "video/mp4")
return "hvc1.1.6.L93.B0";
#endif // BUILDFLAG(ENABLE_PLATFORM_HEVC) &&
// BUILDFLAG(USE_CHROMEOS_PROTECTED_MEDIA)
if (codecs == "" && content_type == "audio/aac") if (codecs == "" && content_type == "audio/aac")
return "aac"; return "aac";
if (codecs == "" && if (codecs == "" &&
......
...@@ -16,6 +16,7 @@ ...@@ -16,6 +16,7 @@
#include "media/filters/chunk_demuxer.h" #include "media/filters/chunk_demuxer.h"
#include "media/filters/frame_processor.h" #include "media/filters/frame_processor.h"
#include "media/filters/source_buffer_stream.h" #include "media/filters/source_buffer_stream.h"
#include "media/media_buildflags.h"
namespace media { namespace media {
...@@ -714,6 +715,29 @@ bool SourceBufferState::OnNewConfigs( ...@@ -714,6 +715,29 @@ bool SourceBufferState::OnNewConfigs(
<< " config: " << video_config.AsHumanReadableString(); << " config: " << video_config.AsHumanReadableString();
DCHECK(video_config.IsValidConfig()); DCHECK(video_config.IsValidConfig());
if (video_config.codec() == kCodecHEVC) {
#if BUILDFLAG(ENABLE_PLATFORM_HEVC)
#if BUILDFLAG(USE_CHROMEOS_PROTECTED_MEDIA)
// On ChromeOS, HEVC is only supported through EME, so require the
// config to be for an encrypted track if on ChromeOS.
// BIG TODO: Is there a test need for conditionally allowing clear HEVC
// here if kEnableClearHevcForTesting in cmdline?
if (video_config.encryption_scheme() ==
EncryptionScheme::kUnencrypted) {
MEDIA_LOG(ERROR, media_log_)
<< "MSE playback of HEVC on ChromeOS is only supported via "
"platform decryptor, but the provided HEVC track is not "
"encrypted.";
return false;
}
#endif // BUILDFLAG(USE_CHROMEOS_PROTECTED_MEDIA)
#else
NOTREACHED()
<< "MSE parser must not emit HEVC tracks on build configurations "
"that do not support HEVC playback via platform.";
#endif // BUILDFLAG(ENABLE_PLATFORM_HEVC)
}
const auto& it = std::find(expected_vcodecs.begin(), const auto& it = std::find(expected_vcodecs.begin(),
expected_vcodecs.end(), video_config.codec()); expected_vcodecs.end(), video_config.codec());
if (it == expected_vcodecs.end()) { if (it == expected_vcodecs.end()) {
......
...@@ -457,6 +457,24 @@ static SupportsType CheckTypeAndCodecs( ...@@ -457,6 +457,24 @@ static SupportsType CheckTypeAndCodecs(
ParserFactoryFunction* factory_function, ParserFactoryFunction* factory_function,
std::vector<CodecInfo::HistogramTag>* audio_codecs, std::vector<CodecInfo::HistogramTag>* audio_codecs,
std::vector<CodecInfo::HistogramTag>* video_codecs) { std::vector<CodecInfo::HistogramTag>* video_codecs) {
#if BUILDFLAG(USE_PROPRIETARY_CODECS) && BUILDFLAG(ENABLE_PLATFORM_HEVC) && \
BUILDFLAG(USE_CHROMEOS_PROTECTED_MEDIA)
// TODO(crbug.com/535738): Undo this special-casing for just EME+HEVC+ChromeOS
// relaxed codec-specificity when widening the relaxation, while maintaining
// requirement of platform support for all detected tracks in SouceBufferState
// handling of parser new-configs callback.
if (type == "video/mp4" && codecs.empty()) {
// Assume EME+HEVC track will be included. Signal only MayBeSupported,
// because isTypeSupported(...) should still require full codec-specificity.
if (factory_function)
*factory_function = &BuildMP4Parser;
if (video_codecs)
video_codecs->push_back(CodecInfo::HISTOGRAM_HEVC);
return MayBeSupported;
}
#endif // BUILDFLAG(USE_PROPRIETARY_CODECS) && BUILDFLAG(ENABLE_PLATFORM_HEVC)
// && BUILDFLAG(USE_CHROMEOS_PROTECTED_MEDIA)
// Search for the SupportedTypeInfo for |type|. // Search for the SupportedTypeInfo for |type|.
for (size_t i = 0; i < base::size(kSupportedTypeInfo); ++i) { for (size_t i = 0; i < base::size(kSupportedTypeInfo); ++i) {
const SupportedTypeInfo& type_info = kSupportedTypeInfo[i]; const SupportedTypeInfo& type_info = kSupportedTypeInfo[i];
...@@ -540,18 +558,34 @@ std::unique_ptr<StreamParser> StreamParserFactory::Create( ...@@ -540,18 +558,34 @@ std::unique_ptr<StreamParser> StreamParserFactory::Create(
std::vector<CodecInfo::HistogramTag> audio_codecs; std::vector<CodecInfo::HistogramTag> audio_codecs;
std::vector<CodecInfo::HistogramTag> video_codecs; std::vector<CodecInfo::HistogramTag> video_codecs;
if (IsSupported == CheckTypeAndCodecs(type, codecs, media_log, // TODO(crbug.com/535738): Fully relax the requirement for specific codecs
&factory_function, &audio_codecs, // (allow MayBeSupported here), and relocate the logging to the parser
&video_codecs)) { // configuration callback. This creation method is called in AddId(), and also
// in CanChangeType() and ChangeType(), so potentially overlogs codecs leading
// to disproportion versus actually parsed codec configurations from
// initialization segments. For this work and also recording when implicit
// codec switching occurs (without explicit ChangeType), see
// https://crbug.com/535738.
SupportsType supportsType = CheckTypeAndCodecs(
type, codecs, media_log, &factory_function, &audio_codecs, &video_codecs);
#if BUILDFLAG(USE_PROPRIETARY_CODECS) && BUILDFLAG(ENABLE_PLATFORM_HEVC) && \
BUILDFLAG(USE_CHROMEOS_PROTECTED_MEDIA)
// TODO(crbug.com/535738): Undo this special-casing when widening relaxation
// support beyond EME+HEVC+ChromeOS.
if (supportsType == MayBeSupported && type == "video/mp4" && codecs.empty()) {
DCHECK(audio_codecs.empty());
DCHECK_EQ(video_codecs.size(), 1U);
DCHECK_EQ(video_codecs[0], CodecInfo::HISTOGRAM_HEVC);
// Override the addSourceBuffer(), canChangeType() and changeType() result
// for just "video/mp4" in this case to be IsSupported locally here, to let
// the actual parser creation proceed.
supportsType = IsSupported;
}
#endif // BUILDFLAG(USE_PROPRIETARY_CODECS) && BUILDFLAG(ENABLE_PLATFORM_HEVC)
// && BUILDFLAG(USE_CHROMEOS_PROTECTED_MEDIA)
if (IsSupported == supportsType) {
// Log the expected codecs. // Log the expected codecs.
// TODO(wolenetz): Relax the requirement for specific codecs (allow
// MayBeSupported here), and relocate the logging to the parser
// configuration callback. This creation method is called in AddId(), and
// also in CanChangeType() and ChangeType(), so potentially overlogs codecs
// leading to disproportion versus actually parsed codec configurations from
// initialization segments. For this work and also recording when implicit
// codec switching occurs (without explicit ChangeType), see
// https://crbug.com/535738.
for (size_t i = 0; i < audio_codecs.size(); ++i) { for (size_t i = 0; i < audio_codecs.size(); ++i) {
UMA_HISTOGRAM_ENUMERATION("Media.MSE.AudioCodec", audio_codecs[i], UMA_HISTOGRAM_ENUMERATION("Media.MSE.AudioCodec", audio_codecs[i],
CodecInfo::HISTOGRAM_MAX + 1); CodecInfo::HISTOGRAM_MAX + 1);
......
...@@ -2,6 +2,7 @@ ...@@ -2,6 +2,7 @@
# Use of this source code is governed by a BSD-style license that can be # Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file. # found in the LICENSE file.
import("//media/media_options.gni")
import("//third_party/blink/renderer/modules/modules.gni") import("//third_party/blink/renderer/modules/modules.gni")
blink_modules_sources("mediasource") { blink_modules_sources("mediasource") {
......
...@@ -7,6 +7,7 @@ include_rules = [ ...@@ -7,6 +7,7 @@ include_rules = [
"+media/base/video_decoder_config.h", "+media/base/video_decoder_config.h",
"+media/filters", "+media/filters",
"+media/formats/mp4/box_definitions.h", "+media/formats/mp4/box_definitions.h",
"+media/media_buildflags.h",
"+third_party/blink/renderer/modules/event_modules.h", "+third_party/blink/renderer/modules/event_modules.h",
"+third_party/blink/renderer/modules/event_target_modules.h", "+third_party/blink/renderer/modules/event_target_modules.h",
"+third_party/blink/renderer/modules/mediasource", "+third_party/blink/renderer/modules/mediasource",
......
...@@ -11,6 +11,7 @@ ...@@ -11,6 +11,7 @@
#include "media/base/audio_decoder_config.h" #include "media/base/audio_decoder_config.h"
#include "media/base/logging_override_if_enabled.h" #include "media/base/logging_override_if_enabled.h"
#include "media/base/video_decoder_config.h" #include "media/base/video_decoder_config.h"
#include "media/media_buildflags.h"
#include "third_party/blink/public/common/privacy_budget/identifiability_metric_builder.h" #include "third_party/blink/public/common/privacy_budget/identifiability_metric_builder.h"
#include "third_party/blink/public/common/privacy_budget/identifiability_study_settings.h" #include "third_party/blink/public/common/privacy_budget/identifiability_study_settings.h"
#include "third_party/blink/public/common/privacy_budget/identifiable_surface.h" #include "third_party/blink/public/common/privacy_budget/identifiable_surface.h"
...@@ -189,13 +190,11 @@ SourceBuffer* MediaSource::addSourceBuffer(const String& type, ...@@ -189,13 +190,11 @@ SourceBuffer* MediaSource::addSourceBuffer(const String& type,
// 2. If type contains a MIME type that is not supported ..., then throw a // 2. If type contains a MIME type that is not supported ..., then throw a
// NotSupportedError exception and abort these steps. // NotSupportedError exception and abort these steps.
// // TODO(crbug.com/535738): Increase relaxation of codec-specificity beyond
// TODO(wolenetz): Refactor and use a less-strict version of isTypeSupported // initial special-casing.
// here. As part of that, CreateWebSourceBuffer in Chromium should inherit if (!IsTypeSupportedInternal(
// relaxation of impl's StreamParserFactory (since it returns false if a GetExecutionContext(), type,
// stream parser can't be constructed with |type|). See false /* Allow underspecified codecs in |type| */)) {
// https://crbug.com/535738.
if (!isTypeSupported(GetExecutionContext(), type)) {
LogAndThrowDOMException( LogAndThrowDOMException(
exception_state, DOMExceptionCode::kNotSupportedError, exception_state, DOMExceptionCode::kNotSupportedError,
"The type provided ('" + type + "') is unsupported."); "The type provided ('" + type + "') is unsupported.");
...@@ -506,11 +505,23 @@ bool MediaSource::IsUpdating() const { ...@@ -506,11 +505,23 @@ bool MediaSource::IsUpdating() const {
// static // static
bool MediaSource::isTypeSupported(ExecutionContext* context, bool MediaSource::isTypeSupported(ExecutionContext* context,
const String& type) { const String& type) {
bool result = IsTypeSupportedInternal(
context, type, true /* Require fully specified mime and codecs */);
DVLOG(2) << __func__ << "(" << type << ") -> " << (result ? "true" : "false");
return result;
}
// static
bool MediaSource::IsTypeSupportedInternal(ExecutionContext* context,
const String& type,
bool enforce_codec_specificity) {
// Section 2.2 isTypeSupported() method steps. // Section 2.2 isTypeSupported() method steps.
// https://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html#widl-MediaSource-isTypeSupported-boolean-DOMString-type // https://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html#widl-MediaSource-isTypeSupported-boolean-DOMString-type
// 1. If type is an empty string, then return false. // 1. If type is an empty string, then return false.
if (type.IsEmpty()) { if (type.IsEmpty()) {
DVLOG(1) << __func__ << "(" << type << ") -> false (empty input)"; DVLOG(1) << __func__ << "(" << type << ", "
<< (enforce_codec_specificity ? "true" : "false")
<< ") -> false (empty input)";
return false; return false;
} }
...@@ -519,7 +530,9 @@ bool MediaSource::isTypeSupported(ExecutionContext* context, ...@@ -519,7 +530,9 @@ bool MediaSource::isTypeSupported(ExecutionContext* context,
// 2. If type does not contain a valid MIME type string, then return false. // 2. If type does not contain a valid MIME type string, then return false.
if (content_type.GetType().IsEmpty()) { if (content_type.GetType().IsEmpty()) {
DVLOG(1) << __func__ << "(" << type << ") -> false (invalid mime type)"; DVLOG(1) << __func__ << "(" << type << ", "
<< (enforce_codec_specificity ? "true" : "false")
<< ") -> false (invalid mime type)";
return false; return false;
} }
...@@ -529,7 +542,8 @@ bool MediaSource::isTypeSupported(ExecutionContext* context, ...@@ -529,7 +542,8 @@ bool MediaSource::isTypeSupported(ExecutionContext* context,
// HTMLMediaElement knows it cannot play. // HTMLMediaElement knows it cannot play.
if (HTMLMediaElement::GetSupportsType(content_type) == if (HTMLMediaElement::GetSupportsType(content_type) ==
MIMETypeRegistry::kIsNotSupported) { MIMETypeRegistry::kIsNotSupported) {
DVLOG(1) << __func__ << "(" << type DVLOG(1) << __func__ << "(" << type << ", "
<< (enforce_codec_specificity ? "true" : "false")
<< ") -> false (not supported by HTMLMediaElement)"; << ") -> false (not supported by HTMLMediaElement)";
RecordIdentifiabilityMetric(context, type, false); RecordIdentifiabilityMetric(context, type, false);
return false; return false;
...@@ -543,17 +557,40 @@ bool MediaSource::isTypeSupported(ExecutionContext* context, ...@@ -543,17 +557,40 @@ bool MediaSource::isTypeSupported(ExecutionContext* context,
// type, media subtype, and codecs then return false. // type, media subtype, and codecs then return false.
// 6. Return true. // 6. Return true.
// For incompletely specified mime-type and codec combinations, we also return // For incompletely specified mime-type and codec combinations, we also return
// false, complying with the non-normative guidance being incubated for the // false if |enforce_codec_specificity| is true, complying with the
// MSE vNext codec switching feature at // non-normative guidance being incubated for the MSE v2 codec switching
// https://github.com/WICG/media-source/tree/codec-switching. // feature at https://github.com/WICG/media-source/tree/codec-switching.
// TODO(wolenetz): Relaxed codec specificity following similar non-normative // Relaxed codec specificity following similar non-normative guidance is
// guidance will soon be allowed for addSourceBuffer and changeType methods, // allowed for addSourceBuffer and changeType methods, but this strict codec
// but this strict codec specificity is and will be retained for // specificity is and will be retained for isTypeSupported.
// isTypeSupported. See https://crbug.com/535738 MIMETypeRegistry::SupportsType supported =
bool result = MIMETypeRegistry::kIsSupported == MIMETypeRegistry::SupportsMediaSourceMIMEType(content_type.GetType(),
MIMETypeRegistry::SupportsMediaSourceMIMEType( codecs);
content_type.GetType(), codecs);
DVLOG(2) << __func__ << "(" << type << ") -> " << (result ? "true" : "false"); #if BUILDFLAG(ENABLE_PLATFORM_HEVC) && BUILDFLAG(USE_CHROMEOS_PROTECTED_MEDIA)
if (supported == MIMETypeRegistry::kMayBeSupported &&
!enforce_codec_specificity && type == "video/mp4") {
// kMayBeSupported here indicates format is supported, but is lacking
// codec-specificity.
// TODO(crbug.com/535738): Increase actual relaxation of codec-specificity
// for addSourceBuffer and changeType usage beyond this initial
// special-casing for just HEVC-EME-CrOS. For now, precisely "video/mp4"
// with underspecified codecs string is assumed to be supported if the build
// supports EME+HEVC on ChromeOS. The underlying Chromium code will require
// precisely one track, encrypted HEVC, to exist when processing
// initialization segments on behalf of a SourceBuffer created with
// addSourceBuffer(|type|) (or currently resulting from changeType(|type|).
supported = MIMETypeRegistry::kIsSupported;
}
#endif // BUILDFLAG(ENABLE_PLATFORM_HEVC) &&
// BUILDFLAG(USE_CHROMEOS_PROTECTED_MEDIA)
bool result = supported == MIMETypeRegistry::kIsSupported;
DVLOG(2) << __func__ << "(" << type << ", "
<< (enforce_codec_specificity ? "true" : "false") << ") -> "
<< (result ? "true" : "false");
RecordIdentifiabilityMetric(context, type, result); RecordIdentifiabilityMetric(context, type, result);
return result; return result;
} }
...@@ -1450,7 +1487,8 @@ std::unique_ptr<WebSourceBuffer> MediaSource::CreateWebSourceBuffer( ...@@ -1450,7 +1487,8 @@ std::unique_ptr<WebSourceBuffer> MediaSource::CreateWebSourceBuffer(
// then throw a NotSupportedError exception and abort these steps. // then throw a NotSupportedError exception and abort these steps.
LogAndThrowDOMException( LogAndThrowDOMException(
exception_state, DOMExceptionCode::kNotSupportedError, exception_state, DOMExceptionCode::kNotSupportedError,
"The type provided ('" + type + "') is not supported."); "The type provided ('" + type +
"') is not supported for SourceBuffer creation.");
return nullptr; return nullptr;
case WebMediaSource::kAddStatusReachedIdLimit: case WebMediaSource::kAddStatusReachedIdLimit:
DCHECK(!web_source_buffer); DCHECK(!web_source_buffer);
......
...@@ -95,6 +95,17 @@ class MediaSource final : public EventTargetWithInlineData, ...@@ -95,6 +95,17 @@ class MediaSource final : public EventTargetWithInlineData,
LOCKS_EXCLUDED(attachment_link_lock_); LOCKS_EXCLUDED(attachment_link_lock_);
static bool isTypeSupported(ExecutionContext* context, const String& type); static bool isTypeSupported(ExecutionContext* context, const String& type);
// Helper for isTypeSupported, addSourceBuffer and SourceBuffer changeType.
// Set |enforce_codec_specificity| true to require fully specified mime and
// codecs, false otherwise.
// TODO(https://crbug.com/535738): When |enforce_codec_specificity| is set to
// false, then fully relax requirements beyond initial special casing for HEVC
// on ChromeOS with EME in mp4.
static bool IsTypeSupportedInternal(ExecutionContext* context,
const String& type,
bool enforce_codec_specificity);
static bool canConstructInDedicatedWorker(); static bool canConstructInDedicatedWorker();
// Methods needed by a MediaSourceAttachmentSupplement to service operations // Methods needed by a MediaSourceAttachmentSupplement to service operations
......
...@@ -1041,7 +1041,9 @@ void SourceBuffer::ChangeType_Locked( ...@@ -1041,7 +1041,9 @@ void SourceBuffer::ChangeType_Locked(
// here. As part of that, CanChangeType in Chromium should inherit relaxation // here. As part of that, CanChangeType in Chromium should inherit relaxation
// of impl's StreamParserFactory (since it returns true iff a stream parser // of impl's StreamParserFactory (since it returns true iff a stream parser
// can be constructed with |type|). See https://crbug.com/535738. // can be constructed with |type|). See https://crbug.com/535738.
if (!MediaSource::isTypeSupported(GetExecutionContext(), type) || if (!MediaSource::IsTypeSupportedInternal(
GetExecutionContext(), type,
false /* allow underspecified codecs in |type| */) ||
!web_source_buffer_->CanChangeType(content_type.GetType(), codecs)) { !web_source_buffer_->CanChangeType(content_type.GetType(), codecs)) {
MediaSource::LogAndThrowDOMException( MediaSource::LogAndThrowDOMException(
*exception_state, DOMExceptionCode::kNotSupportedError, *exception_state, DOMExceptionCode::kNotSupportedError,
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment