Registering Custom XCTest Observer Breaks Test Logging
| Originator: | benchatelain | ||
| Number: | rdar://25456276 | Date Originated: | 30-Mar-2016 07:49 PM |
| Status: | Resolved | Resolved: | 2016-07-08 |
| Product: | Developer Tools | Product Version: | Xcode 7.3 (7D175) |
| Classification: | Serious Bug | Reproducible: | Always |
Summary: Registering an implementation of the XCTestObservation protocol with [[XCTestObservationCenter sharedTestObservationCenter] addTestObserver:] causes XCTest to not output the progress of nor the summary of a test run. Steps to Reproduce: 1. Clone the example project from https://github.com/phatblat/XCTestObserver 2. Open the .xcodeproj 3. Open the Console pane (View -> Debug Area -> Activate Console) 4. Ensure the console output is visible by selecting the "Show the Console" button in the bottom right-hand corner 5. Run the tests (Product -> Test) 6. Console output will only include a few lines - https://raw.githubusercontent.com/phatblat/XCTestObserver/master/test_output1.log 7. Open the TestObserver.m file in the Xcode editor 8. Comment out line 16 which adds the test observer 9. Run the tests (Product -> Test) 10. Console output will include standard XCTest output with summary of the test run - https://raw.githubusercontent.com/phatblat/XCTestObserver/master/test_output2.log Expected Results: The console output in step 6 will be the same (aside from timestamps) as in step 10. Actual Results: The console output in step 6 is missing all output from the XCTest framework. Regression: I do not believe this issue was not present in Xcode 7.2.1. Notes: The XCTest output is used by several tools (xctool) to monitor test progress and report pass/fail. Without the XCTest output these tools cannot perform this function. The Nimble framework (https://github.com/Quick/Nimble) uses this XCTestObservation API to monitor test progress and to recordFailureWithDescription through the current XCTestCase. https://github.com/Quick/Nimble/blob/master/Sources/Nimble/Adapters/NimbleXCTestHandler.swift#L40-L52 When a test suite using Nimble is executed through xctool, it either stalls or crashes waiting for this output. Version & Build: Xcode 7.3 (7D175), Swift Toolchain: Xcode 7.3 Default OS X 10.11.4 (15E65)
Comments
Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!
Fixed in Xcode 8 (confirmed)
Apple Developer Relations23-Jun-2016 01:54 PM
Please verify this issue with the latest Xcode 8 beta build and update your bug report at https://bugreport.apple.com/ with your results.
Xcode 8 beta (Build: 8S128d) https://developer.apple.com/download/