[Xcode 7][XCUITest] Export test results into format that can be read by Continuous Integration systems
| Originator: | richard.yeh | ||
| Number: | rdar://24232345 | Date Originated: | 1-18-16 |
| Status: | Duplicate of 5758007 (Open) | Resolved: | |
| Product: | Developer Tools | Product Version: | |
| Classification: | Feature (New) | Reproducible: |
Summary:
Currently, Xcode 7 generates a lot of useful UI testing output that appears in Xcode's test reports. However, when the tests are run via xcodebuild (as they usually would be if a Continuous Integration system is used), the only output we get is the XCTRunner text to stdout. The screenshots and plists with the testing steps are available at ~/Library/Developer/Xcode/DerivedData/${app}/Logs/Test , but they are separated into multiple files that are neither human readable nor can be imported by any CI system.
It'd be immensely helpful if there was some sort of standard format for UI Testing output (similar to jUnit output) that can be imported into CI systems. Such a format would be useful for both showing human testers which UI steps test cases failed at, and for CI systems to record where points of failure occurred at.
Presently, XCUITest has a lot of potential for running ad hoc automated tests locally, but relies on the Xcode GUI, which diminishes usability as part of CI.
Comments
Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!