Open Radar

 
Summary:
There is a 100% reproducible deadlock in VideoToolbox when suspending the app to background. VTDecompressionSessionDecodeFrame hangs forever waiting on a condition  variable.

We provide a sample project reproducing the issue. It depends on ffmpeg for demuxing, whose libraries we provide pre-compiled for arm64 and armv7.
Use of the VideotoolBox API can be found in sxplayer/decoder_vt.c  The overall architecture of the module is descrived in sxplayer/README.md. 


A few additional information: 

 - This happens only when decodeFlags is set to  kVTDecodeFrame_EnableAsynchronousDecompression and we are not epxlicitely calling VTDecompressionSessionWaitForAsynchronousFrames after the call to VTDecompressionSessionDecodeFrame

- Limiting the internal queue to less than 10 packets does not affect the outcome. 

- Invalidating the session (via VTDecompressionSessionInvalidate) while the call is hanging does not resolve the issue either

- After 10 seconds, there does seem to have a timeout mechanism entering into effect, and our decode callback is called for as much time as there is packets in the queue. However, the previous points still apply

Steps to Reproduce:
1. Create a VTDecompressionSession
2. Setup a callback to consume output frames
3. Push packets via VTDecompressionSessionDecodeFrame in async mode 
4. While pushing packets, put the app in `inactive` state (either by pressing the home button to the homescreen, or by pressing the lock button to the lock screen)


Expected Results:
5. The call to VTDecompressionSessionDecodeFrame eitherfails with a sensible error or succeeds when the app is back in active state


Actual Results:
5. The call to VTDecompressionSessionDecodeFrame hangs forever, regardless of whether or not the application goes back to active state


Version:
iOS 9.0 and above

Notes:
You can simply reproduce the issue by launching our test project, waiting a few seconds for decode to happen, and pressing the lock button. By pausing the program in the debugger, you should notice the deadlock

Note: The chromium team does seem to have stumbled upon the same problem: https://bugs.chromium.org/p/webrtc/issues/detail?id=5464

Configuration:
All iPhone models from the 5 up