Irrelevant crash reports are included in DerivedData/Logs/Attachments/ when running UI tests in parallel on multiple simulators

Originator:pohilets
Number:rdar://39753004 Date Originated:26 Apr 2018
Status:Open Resolved:
Product:Developer Tools Product Version:Xcode 9.2 (9C40b), Xcode 9.3 (9E145), Xcode 9.4b (9Q1004a)
Classification:Bug Reproducible:Always
 
If SUT crashes in one of the invocations of xcodebuild, it will be recorded in all invocations runnings at that moment.

When collecting crash logs, test manager should check for device ID and/or PIDs recorded in the crash log and report only relevant crash logs.

Steps to Reproduce:
1. Check out https://github.com/nickolas-pohilets/rdar-39753004
2. Create two or more simulators of the same configuration (iPhone 6, iOS 11.3)
3. Execute run.sh passing ID's of the simulator devices

Expected Results:
Each of the $DEVICE_ID/Logs/Attachments should contain 4 crash reports, which occurred on that device only.

Actual Results:
Each of the $DEVICE_ID/Logs/Attachments contains (almost) all of the crash reports regardless of the device they occurred on.

Version/Build:
Xcode 9.2 (9C40b), Xcode 9.3 (9E145), Xcode 9.4b (9Q1004a)

Configuration:
macOS 10.13.4 (17E199)

Comments


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!