MSE-in-Workers: Cross-thread function and tests

Includes implementation of CrossThreadMediaSourceAttachment (CTMSA) and
updates to MediaSource and SourceBuffer to use it.

CTMSA owns a lock that it takes on most operations. It also provides a
utility method (RunExclusively()), which is used by the MSE API to take
that same lock during most operations on the API. The same thread
attachment provides a basic version of the same, though no lock is used.

CTMSA understands whether or not it is attached, has ever started
closing, and whether or not either of the attached contexts has ever
destructed. It conditions cross-thread operations to safely behave when
one or both contexts is being destructed.

MediaSource is given its own small lock that it uses to guard its
reference to the attachment instance, when attached. This is required
because attachment start is synchronous on the main thread, even if the
MediaSource is owned by worker context. CTMSA and MediaSource cooperate
to ensure mutex and safety of the two-stage attachment start. In
MediaSource::ContextDestroyed(), further care is taken to prevent
attachment start success when worker context destruction is possibly
racing the main thread's attempt to start attachment.
`context_already_destroyed_`, protected by MediaSource's lock, is used
to prevent that start from succeeding.

MediaSource's context destruction can no longer always assume that
accessing the demuxer via the WebMediaSource (and WebSourceBuffers) is
safe. The worker version of MediaSource context destruction uses the
attachment's RunExclusively() to safely know if it can cleanly Close the
underlying demuxer, or if instead it must do a simpler shutdown.

Future specification work will likely determine some signal to the media
element when the worker-owned MediaSource's context is shutdown, yet the
element itself is still alive. For now, sane results are returned by the
attachment (nothing buffered nor seekable), with no other error
provided. Possible app workarounds might include main and worker
watchdogs and being careful about when the main thread explicitly
terminates the worker.

This experimental implementation attempts to retain
BackgroundVideoOptimization, even for MSE-in-Workers, but does not
support AudioVideoTracks in the worker MediaSource or SourceBuffer.
Plumbing of appropriately-identified track metadata parsed from
initialization segment in the worker is used to populate (and remove)
media element AudioVideoTracks that should agree with the track id's
used internally in WebMediaPlayerImpl to accomplish
BackgroundVideoOptimization.

As a simplification, CTMSA assumes it can be used for at most one
attachment. Effectively, this assumes that the currently on-by-default
MediaSourceRevokeObjectURLOnAttach feature behavior will be always-on
soon. If CTMSA is not unregistered automatically after successful
attachment start (e.g., if that feature is disabled) and if the app
attempts to re-use the corresponding object URL for a later attachment,
that attachment will be rejected by CTMSA. Note that the same-thread
attachment would still allow such re-use when that feature is disabled.

Updated web-test:
  mediasource-worker-attach.html changed to ...-play.html and fixed
New web-test:
  mediasource-worker-play-terminate-worker.html
Manual test: page refreshing while running mediasource-worker-play*.html

BUG=878133

Change-Id: I21f6542b90d51bdc28096500fb1441d202ab4ee8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2459431
Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org>
Reviewed-by: Will Cassella <cassew@google.com>
Cr-Commit-Position: refs/heads/master@{#818174}

third_party/blink/renderer/modules/mediasource/media_source_attachment_supplement.cc

@@ -32,4 +32,23 @@ void MediaSourceAttachmentSupplement::AddMainThreadVideoTrackToMediaElement(
   NOTIMPLEMENTED();
 }
 
+bool MediaSourceAttachmentSupplement::RunExclusively(
+    bool /* abort_if_not_fully_attached */,
+    RunExclusivelyCB cb) {
+  std::move(cb).Run(ExclusiveKey());
+  return true;  // Indicates that we ran |cb|.
+}
+
+void MediaSourceAttachmentSupplement::
+    AssertCrossThreadMutexIsAcquiredForDebugging() {
+  DCHECK(false)
+      << "This should only be called on a CrossThreadMediaSourceAttachment";
+}
+
+// protected
+MediaSourceAttachmentSupplement::ExclusiveKey
+MediaSourceAttachmentSupplement::GetExclusiveKey() const {
+  return ExclusiveKey();
+}
+
 }  // namespace blink

third_party/blink/renderer/modules/mediasource/media_source_attachment_supplement.h

@@ -5,13 +5,14 @@
 #ifndef THIRD_PARTY_BLINK_RENDERER_MODULES_MEDIASOURCE_MEDIA_SOURCE_ATTACHMENT_SUPPLEMENT_H_
 #define THIRD_PARTY_BLINK_RENDERER_MODULES_MEDIASOURCE_MEDIA_SOURCE_ATTACHMENT_SUPPLEMENT_H_
 
+#include "base/util/type_safety/pass_key.h"
 #include "third_party/blink/renderer/core/html/media/media_source_attachment.h"
 #include "third_party/blink/renderer/core/html/media/media_source_tracer.h"
 #include "third_party/blink/renderer/core/html/track/audio_track.h"
 #include "third_party/blink/renderer/core/html/track/video_track.h"
-#include "third_party/blink/renderer/modules/mediasource/media_source.h"
 #include "third_party/blink/renderer/platform/heap/handle.h"
 #include "third_party/blink/renderer/platform/wtf/forward.h"
+#include "third_party/blink/renderer/platform/wtf/functional.h"
 
 namespace blink {
 
@@ -25,13 +26,18 @@ class VideoTrackList;
 // members common to all concrete attachments.
 class MediaSourceAttachmentSupplement : public MediaSourceAttachment {
  public:
+  using ExclusiveKey = util::PassKey<MediaSourceAttachmentSupplement>;
+  using RunExclusivelyCB = base::OnceCallback<void(ExclusiveKey)>;
+
   // Communicates a change in the media resource duration to the attached media
   // element. In a same-thread attachment, communicates this information
-  // synchronously. In a cross-thread attachment, communicates asynchronously to
-  // the media element. Same-thread synchronous notification here is primarily
-  // to preserve compliance of API behavior when not using MSE-in-Worker
-  // (setting MediaSource.duration should be synchronously in agreement with
-  // subsequent retrieval of MediaElement.duration, all on the main thread).
+  // synchronously. In a cross-thread attachment, underlying WebMediaSource
+  // should already be asynchronously communicating this information to the
+  // media element, so attachment operation is a no-op. Same-thread synchronous
+  // notification here is primarily to preserve compliance of API behavior when
+  // not using MSE-in-Worker (setting MediaSource.duration should be
+  // synchronously in agreement with subsequent retrieval of
+  // MediaElement.duration, all on the main thread).
   virtual void NotifyDurationChanged(MediaSourceTracer* tracer,
                                      double duration) = 0;
 
@@ -90,10 +96,33 @@ class MediaSourceAttachmentSupplement : public MediaSourceAttachment {
 
   virtual void OnMediaSourceContextDestroyed() = 0;
 
+  // Default is to just run the cb (e.g., for same-thread implementation of the
+  // attachment, since both the media element and the MSE API operate on the
+  // same thread and no mutex is required.) Cross-thread implementation will
+  // first take a lock, then run the cb conditionally on
+  // |abort_if_not_fully_attached|, then release a lock (all synchronously on
+  // the same thread.) Any return values needed by the caller should be passed
+  // by pointer, aka as "out" arguments. For cross-thread case, see further
+  // detail in it's override declaration and implementation. PassKey pattern
+  // usage (with ExclusiveKey instance passed to |cb| when running it)
+  // statically ensures that only a MediaSourceAttachmentSupplement
+  // implementation can run such a |cb|. |cb| can then be assured that it is run
+  // within the scope of this method.
+  virtual bool RunExclusively(bool abort_if_not_fully_attached,
+                              RunExclusivelyCB cb);
+
+  // Default implementation fails DCHECK. See CrossThreadMediaSourceAttachment
+  // for override. MediaSource and SourceBuffer use this to help verify they
+  // only use underlying demuxer in cross-thread debug case while the
+  // cross-thread mutex is held.
+  virtual void AssertCrossThreadMutexIsAcquiredForDebugging();
+
  protected:
   MediaSourceAttachmentSupplement();
   ~MediaSourceAttachmentSupplement() override;
 
+  ExclusiveKey GetExclusiveKey() const;
+
   DISALLOW_COPY_AND_ASSIGN(MediaSourceAttachmentSupplement);
 };
 

third_party/blink/renderer/modules/mediasource/same_thread_media_source_attachment.cc

@@ -244,7 +244,14 @@ void SameThreadMediaSourceAttachment::CompleteAttachingToMediaElement(
 void SameThreadMediaSourceAttachment::Close(MediaSourceTracer* tracer) {
   // The media element may have already notified us that its context is
   // destroyed, so VerifyCalledWhileContextIsAliveForDebugging() is unusable in
-  // this scope.
+  // this scope. Note that we might be called during main thread context
+  // destruction, and the ordering of MediaSource versus HTMLMediaElement
+  // context destruction notification is nondeterminate. MediaSource closes
+  // itself on its context destruction notification, so elide calling Close()
+  // again on it in that case. This affords stronger guarantees on when
+  // MediaSource::Close is callable by an attachment.
+  if (media_source_context_destroyed_)
+    return;
 
   GetMediaSource(tracer)->Close();
 }
@@ -253,14 +260,24 @@ WebTimeRanges SameThreadMediaSourceAttachment::BufferedInternal(
     MediaSourceTracer* tracer) const {
   VerifyCalledWhileContextsAliveForDebugging();
 
-  return GetMediaSource(tracer)->BufferedInternal();
+  // Since the attached MediaSource, HTMLMediaElement and the element's player's
+  // underlying demuxer are all owned by the main thread in this SameThread
+  // attachment, it is safe to get an ExclusiveKey here to pass to the attached
+  // MediaSource to let it know it is safe to access the underlying demuxer to
+  // tell us what is currently buffered.
+  return GetMediaSource(tracer)->BufferedInternal(GetExclusiveKey());
 }
 
 WebTimeRanges SameThreadMediaSourceAttachment::SeekableInternal(
     MediaSourceTracer* tracer) const {
   VerifyCalledWhileContextsAliveForDebugging();
 
-  return GetMediaSource(tracer)->SeekableInternal();
+  // Since the attached MediaSource, HTMLMediaElement and the element's player's
+  // underlying demuxer are all owned by the main thread in this SameThread
+  // attachment, it is safe to get an ExclusiveKey here to pass to the attached
+  // MediaSource to let it know it is safe to access the underlying demuxer to
+  // tell us what is currently seekable.
+  return GetMediaSource(tracer)->SeekableInternal(GetExclusiveKey());
 }
 
 void SameThreadMediaSourceAttachment::OnTrackChanged(MediaSourceTracer* tracer,

third_party/blink/renderer/modules/mediasource/source_buffer.h

@@ -39,6 +39,7 @@
 #include "third_party/blink/renderer/core/execution_context/execution_context_lifecycle_observer.h"
 #include "third_party/blink/renderer/core/typed_arrays/array_buffer_view_helpers.h"
 #include "third_party/blink/renderer/modules/event_target_modules.h"
+#include "third_party/blink/renderer/modules/mediasource/media_source_attachment_supplement.h"
 #include "third_party/blink/renderer/modules/mediasource/track_default_list.h"
 #include "third_party/blink/renderer/platform/scheduler/public/post_cancellable_task.h"
 #include "third_party/blink/renderer/platform/weborigin/kurl.h"
@@ -77,7 +78,6 @@ class SourceBuffer final : public EventTargetWithInlineData,
   void setMode(const AtomicString&, ExceptionState&);
   bool updating() const { return updating_; }
   TimeRanges* buffered(ExceptionState&) const;
-  WebTimeRanges buffered() const;
   double timestampOffset() const;
   void setTimestampOffset(double, ExceptionState&);
   void appendBuffer(DOMArrayBuffer* data, ExceptionState&);
@@ -100,6 +100,18 @@ class SourceBuffer final : public EventTargetWithInlineData,
   AudioTrackList& audioTracks();
   VideoTrackList& videoTracks();
 
+  // "_Locked" requires these be called while in the scope of callback of
+  // |source_|'s attachment's RunExclusively(). Other methods without "_Locked"
+  // may also require the same, since they can be called from within these
+  // methods.
+  void SetMode_Locked(
+      AtomicString,
+      ExceptionState*,
+      MediaSourceAttachmentSupplement::ExclusiveKey /* passkey */);
+  void GetBuffered_Locked(
+      WebTimeRanges* /* out */,
+      MediaSourceAttachmentSupplement::ExclusiveKey /* passkey */) const;
+
   void RemovedFromMediaSource();
   double HighestPresentationTimestamp();
 
@@ -127,12 +139,9 @@ class SourceBuffer final : public EventTargetWithInlineData,
 
   bool PrepareAppend(double media_time, size_t new_data_size, ExceptionState&);
   bool EvictCodedFrames(double media_time, size_t new_data_size);
-  void AppendBufferInternal(double media_time,
-                            const unsigned char*,
-                            size_t,
-                            ExceptionState&);
+  void AppendBufferInternal(const unsigned char*, size_t, ExceptionState&);
   void AppendBufferAsyncPart();
-  void AppendError();
+  void AppendError(MediaSourceAttachmentSupplement::ExclusiveKey /* passkey */);
 
   void RemoveAsyncPart();
 
@@ -141,6 +150,41 @@ class SourceBuffer final : public EventTargetWithInlineData,
 
   void RemoveMediaTracks();
 
+  // "_Locked" requires these be called while in the scope of callback of
+  // |source_|'s attachment's RunExclusively(). Other methods without "_Locked"
+  // may also require the same, since they can be called from within these
+  // methods.
+  void SetTimestampOffset_Locked(
+      double,
+      ExceptionState*,
+      MediaSourceAttachmentSupplement::ExclusiveKey /* passkey */);
+  void SetAppendWindowStart_Locked(
+      double,
+      MediaSourceAttachmentSupplement::ExclusiveKey /* passkey */);
+  void SetAppendWindowEnd_Locked(
+      double,
+      MediaSourceAttachmentSupplement::ExclusiveKey /* passkey */);
+  void Abort_Locked(
+      MediaSourceAttachmentSupplement::ExclusiveKey /* passkey */);
+  void Remove_Locked(
+      double start,
+      double end,
+      ExceptionState*,
+      MediaSourceAttachmentSupplement::ExclusiveKey /* passkey */);
+  void ChangeType_Locked(
+      const String& type,
+      ExceptionState*,
+      MediaSourceAttachmentSupplement::ExclusiveKey /* passkey */);
+  void AppendBufferInternal_Locked(
+      const unsigned char*,
+      size_t,
+      ExceptionState*,
+      MediaSourceAttachmentSupplement::ExclusiveKey /* passkey */);
+  void AppendBufferAsyncPart_Locked(
+      MediaSourceAttachmentSupplement::ExclusiveKey /* passkey */);
+  void RemoveAsyncPart_Locked(
+      MediaSourceAttachmentSupplement::ExclusiveKey /* passkey */);
+
   // Returns MediaElement playback position (i.e. MediaElement.currentTime() )
   // in seconds, or NaN if media element is not available.
   double GetMediaTime();

third_party/blink/web_tests/external/wpt/media-source/dedicated-worker/mediasource-worker-play-terminate-worker.html

+<!DOCTYPE html>
+<html>
+<title>MediaSource-in-Worker looped playback test case with worker termination at various places</title>
+<script src="/resources/testharness.js"></script>
+<script src="/resources/testharnessreport.js"></script>
+<body>
+<script>
+
+function terminateWorkerAfterMultipleSetTimeouts(test, worker, timeouts_remaining) {
+  if (timeouts_remaining <= 0) {
+    worker.terminate();
+    test.step_timeout(() => { test.done(); }, 0);
+  } else {
+    test.step_timeout(() => {
+      terminateWorkerAfterMultipleSetTimeouts(test, worker, --timeouts_remaining);
+    }, 0);
+  }
+}
+
+function startWorkerAndTerminateWorker(test, when_to_start_timeouts, timeouts_to_await) {
+  const worker = new Worker("mediasource-worker-util.js");
+  worker.onerror = test.unreached_func("worker error");
+
+  const video = document.createElement("video");
+  document.body.appendChild(video);
+  video.onerror = test.unreached_func("video element error");
+
+  if (when_to_start_timeouts == "after first ended event") {
+    video.addEventListener("ended", test.step_func(() => {
+      terminateWorkerAfterMultipleSetTimeouts(test, worker, timeouts_to_await);
+      video.currentTime = 0;
+      video.loop = true;
+    }), { once : true });
+  } else {
+    video.loop = true;
+  }
+
+  if (when_to_start_timeouts == "before setting src") {
+    terminateWorkerAfterMultipleSetTimeouts(test, worker, timeouts_to_await);
+  }
+
+  worker.onmessage = test.step_func((e) => {
+    if (e.data.substr(0,6) == "Error:") {
+      assert_unreached("Worker error: " + e.data);
+    } else {
+      const url = e.data
+      assert_true(url.match(/^blob:.+/) != null);
+      video.src = url;
+      if (when_to_start_timeouts == "after setting src") {
+        terminateWorkerAfterMultipleSetTimeouts(test, worker, timeouts_to_await);
+      }
+      video.play().catch(error => {
+        // Only rejections due to MEDIA_ERR_SRC_NOT_SUPPORTED are expected to possibly
+        // occur, except if we expect to reach at least 1 'ended' event.
+        assert_not_equals(when_to_start_timeouts, "after first ended event");
+        assert_true(video.error != null);
+        assert_equals(video.error.code, MediaError.MEDIA_ERR_SRC_NOT_SUPPORTED);
+        // Do not rethrow. Instead, wait for the step_timeouts to finish the test.
+      });
+    }
+  });
+}
+
+[ "before setting src", "after setting src", "after first ended event" ].forEach((when) => {
+  for (let timeouts = 0; timeouts < 10; ++timeouts) {
+    async_test((test) => { startWorkerAndTerminateWorker(test, when, timeouts); },
+        "Test worker MediaSource termination after at least " + timeouts +
+          " main thread setTimeouts, starting counting " + when);
+  }
+});
+
+</script>
+</body>
+</html>

third_party/blink/web_tests/external/wpt/media-source/dedicated-worker/mediasource-worker-attach.html → third_party/blink/web_tests/external/wpt/media-source/dedicated-worker/mediasource-worker-play.html

 <!DOCTYPE html>
 <html>
-<title>Test attachment to dedicated worker MediaSource</title>
+<title>Simple MediaSource-in-Worker playback test case</title>
 <script src="/resources/testharness.js"></script>
 <script src="/resources/testharnessreport.js"></script>
 <body>
@@ -9,15 +9,8 @@
 async_test((t) => {
   const video = document.createElement('video');
   document.body.appendChild(video);
-
-  // TODO(https://crbug.com/878133): Enable attachment success by
-  // completing the CrossThreadAttachment implementation. Currently,
-  // a custom Chromium MediaError.message is confirmed.
-  video.onerror = t.step_func(() => {
-    assert_not_equals(video.error, null);
-    assert_equals(video.error.message, "MEDIA_ELEMENT_ERROR: Unable to attach MediaSource");
-    t.done();
-  });
+  video.onerror = t.unreached_func("video element error");
+  video.onended = t.step_func_done();
 
   let worker = new Worker("mediasource-worker-util.js");
   worker.onerror = t.unreached_func("worker error");
@@ -28,9 +21,10 @@ async_test((t) => {
       const url = e.data;
       assert_true(url.match(/^blob:.+/) != null);
       video.src = url;
+      video.play();
     }
   });
-}, "Test worker MediaSource attachment (currently should fail to attach)");
+}, "Test worker MediaSource construction, attachment, buffering and basic playback");
 
 // TODO(https://crbug.com/878133): Test multiple attachments to same worker
 // MediaSource racing each other: precisely one should win the race.

third_party/blink/web_tests/external/wpt/media-source/dedicated-worker/mediasource-worker-util.js

@@ -13,11 +13,11 @@ let mediaLoad;
 // Find supported test media, if any.
 let MEDIA_LIST = [
   {
-    url: 'mp4/test.mp4',
+    url: '../mp4/test.mp4',
     type: 'video/mp4; codecs="mp4a.40.2,avc1.4d400d"',
   },
   {
-    url: 'webm/test.webm',
+    url: '../webm/test.webm',
     type: 'video/webm; codecs="vp8, vorbis"',
   },
 ];
@@ -56,9 +56,6 @@ onmessage = function(evt) {
   postMessage("Error: No message expected by Worker");
 };
 
-// TODO(https://crbug.com/878133): Enable this path by completing the
-// CrossThreadMediaSourceAttachment implementation such that attachment can
-// actually succeed and 'sourceopen' be dispatched.
 mediaSource.addEventListener("sourceopen", () => {
   URL.revokeObjectURL(mediaSourceObjectUrl);
   sourceBuffer = mediaSource.addSourceBuffer(mediaMetadata.type);
@@ -66,10 +63,16 @@ mediaSource.addEventListener("sourceopen", () => {
     postMessage("Error: " + err);
   };
   sourceBuffer.onupdateend = () => {
+    // Reset the parser. Unnecessary for this buffering, except helps with test
+    // coverage.
+    sourceBuffer.abort();
     // Shorten the buffered media and test playback duration to avoid timeouts.
     sourceBuffer.remove(0.5, Infinity);
     sourceBuffer.onupdateend = () => {
       sourceBuffer.duration = 0.5;
+      // Issue changeType to the same type that we've already buffered.
+      // Unnecessary for this buffering, except helps with test coverage.
+      sourceBuffer.changeType(mediaMetadata.type);
       mediaSource.endOfStream();
     };
   };