Open Radar

 
Resolution:
Engineering has determined that there are no plans to address this based on the following:

This is a known issue because the audio from the MUXed device is not coming in through the HAL.  

We are now closing this bug report.


Summary:
The AVSampleBufferDisplayLayer can be used to draw CMSampleBufferRefs (video frames) to a CALayer, which means you can get a capture frame, apply effects, make a new CMSampleBufferRef, and put that onscreen. It's pretty great.

Problem! The layer stops updating if the AVCaptureSession that is supplying CMSampleBuffers starts getting audio from a muxed source, such as a DV camcorder. The effect is to freeze the layer on its last displayed video frame. The problem manifests itself regardless of what the video device is (ie, whether it's the same muxed device being used for the audio, or a USB camera)

Steps to Reproduce:
Run the attached MuxedMicVideoFreezeBuglet project. You will need to have a muxed video device attached (I used an old Canon ZR25 camcorder; we've also seen this problem on a AG-HPX170 DVCPRO HD camera). Make sure OS X can see the camera (ie, bring up QuickTime Player X, do "new video recording", and make sure the camera is selectable… you may need to disconnect and reconnect to see the device [this has been my experience with DV on QTX]).

When the app is running there are two views: on the left is a AVCaptureVideoPreviewLayer, and on the right is a AVSampleBufferDisplayLayer. The view on the right is updated by just enqueing video buffers as they come in to -[AVCaptureVideoDataOutputSampleBufferDelegate captureOutput:didOutputSampleBuffer:fromConnection:]. A label on the right side keeps track of how many callback calls have been made.

Two popup menus allow you to change video and audio input devices. Switch between your cameras and microphones. Each time there will be a brief pause as the capture session is rebuilt, but it will eventually start updating both views again.

Expected Results:
By and large, the two video previews should be the same (perhaps modulo a dropped frame here or there if AVCaptureVideoPreviewLayer is more efficient than my callback).

Actual Results:
This works as expected until and unless you choose a muxed device for your audio input (these devices are shown as "Device Name (muxed)" in the popups). When you do this, the AVSampleBufferDisplayLayer will stop updating. This occurs regardless of what your video device is (ie, whether or not it's the same muxed device you're getting audio from) The AVCaptureVideoPreviewLayer continues to update as normal. The callback count keeps incrementing, meaning that the AVCaptureVideoDataOutputSampleBufferDelegate are still occurring, and setting a breakpoint will show that -[AVSampleBufferDisplayLayer enqueueBuffer:] is still being called… it just doesn't have any visible effect.

A screen recording of the buglet app in action is available at:
https://dl.dropboxusercontent.com/u/12216224/buglets/MuxedMicVideoFreezeBuglet/MuxedMicVideoFreezeBuglet-screenflow.mp4

Version:
OS X 10.9.3, build 13D65

Notes:


Configuration:
Running this on an Early 2008 Mac Pro, using a crappy third-party USB camera, Canon ZR 25 DV camcorder, analog mic for line in, and an Audio Technica USB microphone.

Attachments:
'MuxedMicVideoFreezeBuglet.zip' was successfully uploaded.

-----
Bonus for OpenRadar readers:
Buglet code: https://dl.dropboxusercontent.com/u/12216224/buglets/MuxedMicVideoFreezeBuglet/MuxedMicVideoFreezeBuglet.zip
Demo video: https://dl.dropboxusercontent.com/u/12216224/buglets/MuxedMicVideoFreezeBuglet/MuxedMicVideoFreezeBuglet-screenflow.mp4