xctest bundles should include dependency frameworks

Originator:ruenzuo
Number:rdar://32198867 Date Originated:15-May-2017
Status:Open Resolved:
Product:Developer Tools Product Version:Xcode 8.3.2
Classification:Enhancement Reproducible:
 
Summary:
At WWDC16, during session 409 "Advanced Testing and Continuous Integration", `build-for-testing` and `test-without-building` were introduced. Since then, it has been possible to build test in one computer and move over the build products to run those tests in a different computer.

For testing iOS apps, the procedure is straightforward as the test bundle, frameworks, and other dependencies, are contained within the application bundle, so it's easy to identify the right build product and move it to a different computer.

For frameworks this isn't the case. The produced xctest bundle doesn't know anything about it's dependencies so it's up to the user to find the right frameworks and put them in the build directory so they're found while running the tests.

This becomes a problem if derived data is shared between many framework targets, as looking for "all framework in build directory" could be potentially expensive.

Steps to Reproduce:
Using the latest Xcode version (8.3.2 at the time of writing)
1. Run test_framework.sh

FrameworkZ is a completely unrelated build product, but following that procedure (best one I could came up with) would incorrectly "stash it" as a dependency of the tests.

Expected Results:
Ideally everything necessary to run the unit tests should be contained in "something" (ideally the xctest bundle) so moving a directory between computers becomes feasible. 

Actual Results:
Sharing derived data becomes really hard and for complex setups, build-for-testing and test-without-building are not helpful by themselves. 

Version:
Xcode 8.3.2

Notes:


Configuration:
Xcode 8.3.2, macOS Sierra 10.12.4

Attachments:
'XING.zip' was successfully uploaded.

Comments


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!