Open Radar

 
Radar: FB7122062

The XCTest invocation contains the `CONFIGURATION_BUILD_DIR` path for *all targets in the selected scheme* in a bunch of different environment variables:

* DYLD_FRAMEWORK_PATH
* DYLD_LIBRARY_PATH
* __XCODE_BUILT_PRODUCTS_DIR_PATHS
* __XPC_DYLD_FRAMEWORK_PATH
* __XPC_DYLD_LIBRARY_PATH

We use the `CONFIGURATION_BUILD_DIR` build setting for per-target isolation. We have a single scheme that contains all of our unit tests for all of our modules. This simplifies our CI configuration and makes it easy for developers to run unit tests without needing to switch schemes to the module containing the test.

As we've modularized our codebase, we now have 200+ unit test bundles. We've gotten to a point where the xctest invocation is failing, because we're hitting the max size of the environment data that can be supplied. This manifests in the IDE as an obscure error message "The request to open <target> failed." and detail "The request was denied by service delegate (SBMainWorkspace) for reason: unspecified". Our current workaround for developers is to have them show or create a scheme for just the module containing the test. This isn't a great experience for the developer though.

On investigating this, it appears that this is at least partially caused by information that is unrelated to the test target being invoked being supplied to xctest. The list of paths supplied continues to grow the more we modularize our codebase.

Please list the steps you took to reproduce the issue:
1. Open attached sample project
2. Select scheme "frameworkA" and run tests (CMD+U)
3. Open the xctest session logs (report navigator -> "Log - Test frameworkA" -> expand and double click "writing diagnostic log for test session")
4. Find log line "Spawned executable at path: .../xctest with options:"
5. Inspect the `DYLD_FRAMEWORK_PATH` value
6. Repeat steps 2-5 with scheme "run-all-tests" and open the diagnostic logs specifically for frameworkATests

What did you expect to happen?
The `DYLD_FRAMEWORK_PATH` for frameworkATests to be the same, regardless of it being launched from the "frameworkA" scheme or the "run-all-tests" scheme. Each XCTest invocation should only be supplied the `CONFIGURATION_BUILD_DIR` values needed for the specific XCTest bundle being launched.

What actually happened?
The `run-all-tests` scheme frameworkATests xctest invocation contains the build directories for frameworkB, frameworkD, frameworkBTests, frameworkDTests, none of which are dependencies of frameworkATests. Each XCTest invocation is being supplied the `CONFIGURATION_BUILD_DIR` values for *all targets included in the scheme*.