Open Radar

 
Summary:

See this dev forum thread:
https://devforums.apple.com/message/915208#915208

xcodebuild, when run from Jenkins, launches otest but fails to communicate properly with it. This results in unit tests terminating prematurely or not running.

The output log will contain a line such as:

error: failed to launch '/Applications/Xcode.app/Contents/Developer/Tools/otest' -- failed to get the task for process 64298



Steps to Reproduce:

In a shell, perform the following script:

git clone git@github.com:elegantchaos/ECIntegration.git
cd ECIntegration
git submodule update --init --recursive
Scripts/eclogging-tests.sh
ls test-build/reports/*

This should work, and reports should contain some XML files.

Now set up a Jenkins job to do the same thing. It should use the repo git@github.com:elegantchaos/ECIntegration.git and perform the last three lines of the above script.

Expected Results:

When you run the script from the command line, you should get this:

  Setting up tests for ECIntegration
  Building ECLogging Mac for macosx test
  Building ECLogging iOS for iphonesimulator build TEST_AFTER_BUILD=YES
  Building ECLogging Sample Mac for macosx test
  Building ECLogging Sample iOS for iphonesimulator build TEST_AFTER_BUILD=YES
  test-build/reports/iphonesimulator-ECLogging Sample iOS:

  test-build/reports/iphonesimulator-ECLogging iOS:

  test-build/reports/macosx-ECLogging Mac:
  TEST-ECParameterisedTestCase.xml  TEST-StringTests.xml

  test-build/reports/macosx-ECLogging Sample Mac:
  TEST-ECParameterisedTestCase.xml   TEST-RunLoopExample.xml            TEST-category2.xml
  TEST-Included.xml                  TEST-category1.xml                 TEST-parameterisedTestExample.xml


Actual Results:

Running from the shell works.

Running from Jenkins may appear to work, but xcodebuild is actually failing to connect to otest, and is terminating it. 

Because there aren't many tests in this suite, they may complete before xcodebuild terminates otest, but for a longer suite of tests, it will terminate prematurely. If necessary, add some slow tests to ECLogging to verify this.

Examine the logs in: test-build/logs/macosx-ECLogging\ Mac/out.log 

There will be a line:

error: failed to launch '/Applications/Xcode.app/Contents/Developer/Tools/otest' -- failed to get the task for process 64298


Regression:

These tests have been working fine on an old Mac Mini running 10.7 and XC4.6. They do not appear to work on Mavericks + XC5.