Open Radar

 
Increasing `-XCTIDEConnectionTimeout` command line argument is required to avoid timeout "IDE session ready for test plan". Without this, we're hitting a timeout on our CI test runs about 25% of the time. This happened on updating our CI runs to use Xcode 10.1 from Xcode 9.4. 

It happens more on some jobs than others, which leads me to believe there is something that is exacerbating this, but we were unable to isolate what exactly it was. For our cluster, we always create a new clean simulator to start, so it might be taking longer to set up the connection on first simulator launch? 

From looking at the simulator logs, a normal run has:
"Test process runnable PID" -> "Test runner is ready, running protocol 29" -> "Test runner session successfully initiated." (immediately after) (edited)

For the failure case, it has:
"Test runner is ready, running protocol 29" -> (one minute later) -> "Failed to run tests: Timeout waiting for fulfillment of promise for 'IDE session ready for test plan'." ->  "Test process runnable PID"

I'm attaching the logs from a normal case (where the tests ran but a test failed) and the bad case where it timed out trying to establish a session.

I've also found that LinkedIn has independently ran into and worked around this problem in the same way:
https://github.com/linkedin/bluepill/pull/289

Steps to Reproduce:
1. Create a project with a scheme with many targets
2. Add some dynamic frameworks as dependencies of the test bundle
3. Invoke tests from `xcodebuild` CLI with `test-without-building` action

Expected Results:
Tests should start running consistently.

Actual Results:
Sometimes a test target fails to be started before the "IDE session ready" timeout.

Version/Build:
Version 10.1 (10B61)