Skip to content

GitLab

T
tangled
Commit 68154c1d authored by Johannes Kron's avatar Johannes Kron Committed by Commit Bot
Browse files
Options

Add support for generic display rates to WebRTC/low-latency renderer 

The initial implementation of the low-latency renderer assumed
that the display rate was fixed at 60Hz. This CL adds support
for generic frame rates as well as missed render intervals.

The low-latency renderer algorithm is enabled from WebRTC by
setting the metadata field maximum_composition_delay_in_frames.

This is an experimental feature that is active if and only if the
RTP header extension playout-delay is set to min=0ms and max>0ms.

The feature can be completely disabled by specifying the field trial
WebRTC-LowLatencyRenderer/enabled:false/

Bug: 1138888
Change-Id: Ic64b26331d31aaf8fb8ed4fb8f1056b227cb6eaa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2527424Reviewed-by: default avatarGuido Urdaneta <guidou@chromium.org>
Reviewed-by: default avatarJesse Doherty <jwd@chromium.org>
Reviewed-by: default avatarDale Curtis <dalecurtis@chromium.org>
Commit-Queue: Johannes Kron <kron@chromium.org>
Cr-Commit-Position: refs/heads/master@{#830225}
parent 4d61874e
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 653 additions and 120 deletions
third_party/blink/renderer/modules/mediastream/low_latency_video_renderer_algorithm.h
View file @ 68154c1d
......@@ -64,15 +64,27 @@ class MODULES_EXPORT LowLatencyVideoRendererAlgorithm {
}
private:
size_t DetermineModeAndNumberOfFramesToRender(
double fractional_frames_to_render);
bool ReduceSteadyStateQueue(size_t number_of_frames_to_render);
void SelectNextAvailableFrameAndUpdateLastDeadline(
base::TimeTicks deadline_min);
scoped_refptr<media::VideoFrame> current_frame_;
// Queue of incoming frames waiting for rendering.
using VideoFrameQueue = WTF::Deque<scoped_refptr<media::VideoFrame>>;
VideoFrameQueue frame_queue_;
// The length of the last deadline interval given to Render(), updated at the
// start of Render().
base::TimeDelta render_interval_;
// Render deadline min for when the last frame was rendered.
base::Optional<base::TimeTicks> last_render_deadline_min_;
// Stores the number of fractional frames that were not rendered as of
// |last_render_deadline_min_|. This is needed in case the display refresh
// rate is not a multiple of the video stream frame rate.
double unrendered_fractional_frames_;
enum class Mode {
// Render frames at their intended rate.
......@@ -87,6 +99,22 @@ class MODULES_EXPORT LowLatencyVideoRendererAlgorithm {
// The number of consecutive render frames with a post-decode queue back-up
// (defined as greater than one frame).
int consecutive_frames_with_back_up_;
struct Stats {
int total_frames;
int dropped_frames;
int drained_frames;
int render_frame;
int no_new_frame_to_render;
int accumulated_queue_length;
int accumulated_queue_length_count;
int enter_drain_mode;
int reduce_steady_state;
int max_size_drop_queue;
};
Stats stats_;
base::Optional<base::TimeTicks> last_deadline_min_stats_recorded_;
void RecordAndResetStats();
};
} // namespace blink
......
tools/metrics/histograms/histograms_xml/media/histograms.xml
View file @ 68154c1d
......@@ -3042,6 +3042,103 @@ reviews. Googlers can read more about this at go/gwsq-gerrit.
<summary>Video width while remoting content.</summary>
</histogram>
<histogram name="Media.RtcLowLatencyVideoRenderer.AverageQueueLengthX10"
units="frames" expires_after="2021-05-31">
<owner>kron@chromium.org</owner>
<owner>webrtc-video@google.com</owner>
<summary>
10 times the average queue length when Render() is called in the RTC
low-latency video renderer. Repeatedly measured with a period of 100 s for
as long as the stream is active.
</summary>
</histogram>
<histogram name="Media.RtcLowLatencyVideoRenderer.DrainedFramesPermille"
units="permille" expires_after="2021-05-31">
<owner>kron@chromium.org</owner>
<owner>webrtc-video@google.com</owner>
<summary>
Per mille of frames drained in the RTC low-latency video renderer.
Repeatedly measured with a period of 100 s for as long as the stream is
active.
</summary>
</histogram>
<histogram name="Media.RtcLowLatencyVideoRenderer.DroppedFramesPermille"
units="permille" expires_after="2021-05-31">
<owner>kron@chromium.org</owner>
<owner>webrtc-video@google.com</owner>
<summary>
Per mille of frames dropped in the RTC low-latency video renderer.
Repeatedly measured with a period of 100 s for as long as the stream is
active.
</summary>
</histogram>
<histogram name="Media.RtcLowLatencyVideoRenderer.EnterDrainModeCount"
units="count" expires_after="2021-05-31">
<owner>kron@chromium.org</owner>
<owner>webrtc-video@google.com</owner>
<summary>
Number of times drain mode is entered. Repeatedly measured with a period of
100 s for as long as the stream is active.
</summary>
</histogram>
<histogram name="Media.RtcLowLatencyVideoRenderer.MaxSizeDropQueueCount"
units="count" expires_after="2021-05-31">
<owner>kron@chromium.org</owner>
<owner>webrtc-video@google.com</owner>
<summary>
Number of times the maximum queue size is exceeded and the entire queue is
dropped. Repeatedly measured with a period of 100 s for as long as the
stream is active.
</summary>
</histogram>
<histogram name="Media.RtcLowLatencyVideoRenderer.NoNewFrameToRenderPermille"
units="permille" expires_after="2021-05-31">
<owner>kron@chromium.org</owner>
<owner>webrtc-video@google.com</owner>
<summary>
Per mille of render times where the queue is empty and there is no new frame
to render. Repeatedly measured with a period of 100 s for as long as the
stream is active.
</summary>
</histogram>
<histogram name="Media.RtcLowLatencyVideoRenderer.ReduceSteadyStateCount"
units="count" expires_after="2021-05-31">
<owner>kron@chromium.org</owner>
<owner>webrtc-video@google.com</owner>
<summary>
Number of times the reduce steady state queue feature is activated.
Repeatedly measured with a period of 100 s for as long as the stream is
active.
</summary>
</histogram>
<histogram name="Media.RtcLowLatencyVideoRenderer.TotalFrames" units="frames"
expires_after="2021-05-31">
<owner>kron@chromium.org</owner>
<owner>webrtc-video@google.com</owner>
<summary>
Total number of frames enqueued to the RTC low-latency video renderer.
Repeatedly measured with a period of 100 s for as long as the stream is
active.
</summary>
</histogram>
<histogram name="Media.RtcLowLatencyVideoRenderer.TryToRenderFrameCount"
units="count" expires_after="2021-05-31">
<owner>kron@chromium.org</owner>
<owner>webrtc-video@google.com</owner>
<summary>
Number of times we try to render a new frame. Repeatedly measured with a
period of 100 s for as long as the stream is active.
</summary>
</histogram>
<histogram name="Media.RTCVideoDecoderError" enum="VideoDecodeAcceleratorError"
expires_after="M82">
<owner>posciak@chromium.org</owner>
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment