VideoCaptureDeviceMac: Match output and capture format
This relates to the following two classes used when capturing content on macOS: - AVCaptureDevice which does the actual capturing from the device into IOSurfaces - AVCaptureVideoDataOutput takes those IOSurfaces and wrangles them into CMSampleBuffers that are handed to us Bug 1: Not telling AVCaptureDevice our preferences When Chrome wants a specific (width, height, framerate) combination, we request that, along with a pixel format, to AVCaptureVideoDataOutput. While we indicate our preferences to our AVCaptureVideoDataOutput, we do not indicate them to our AVCaptureDevice. As a result, the AVCaptureDevice will often have a completely different format that it's using, and the aforementioned "wrangling" will involve a format conversion and maybe even some downsampling(!). In bad cases, this can burn 10% of a CPU core on its own (fortunately bad cases are rare). Bug 2: Using an unsupported pixel format We currently prefer "422YpCbCr8" aka "UYVY" aka "2vuy", and request it regardless of whether or not it is supported by the capture. Most devices don't actually support it, and so the wrangling starts up again. The fix for these bugs is to trawl through the list of supported pixel formats to find the one that best matches the requested format, and then specify this to our AVCaptureDevice when we start capture. Do this by adding a FindBestCaptureFormat to do the trawling. Also don't prefer the "422YpCbCr8" aka "UYVY" aka "2vuy" format above supported formats. The consequence of this is that many Facetime cameras will now be outputting "422YpCbCr8_yuvs" aka "YUY2" aka "yuvs", and will hopefully use less CPU. When we do get an exact match between input and output, we also gain access to the IOSurface that the camera's data is being thrown into. This could allow a zero copy path all the way through to the compositor. Eventually. Bug: 1124884 Change-Id: I33b4b62e152ba9f3b2be04dc3a5f3084d6ee9438 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2391767 Commit-Queue: ccameron <ccameron@chromium.org> Reviewed-by:Miguel Casas <mcasas@chromium.org> Cr-Commit-Position: refs/heads/master@{#804584}
Showing
Please register or sign in to comment