UI Tests fail running outside app

Originator:t.camin
Number:rdar://27802551 Date Originated:11-08-2016
Status:Duplicate Resolved:
Product:Developer Tools Product Version:
Classification: Reproducible:Always
 
Summary:
We're trying to write a UI Test that verifies that an external link is properly opened outside our app in Safari.app.

We successfully recorded a test that taps on a link and then gathers the URL from Safari's.app address bar. Unfortunately the test fails to run and dies with the following assertion:

Assertion Failure: <unknown>:0: UI Testing Failure - Failure getting snapshot Error Domain=XCTestManagerErrorDomain Code=12 "Failed to get main window after 30 retries: kAXErrorServerNotFound (see <rdar://problem/15530121>)" UserInfo={NSLocalizedDescription=Failed to get main window after 30 retries: kAXErrorServerNotFound (see <rdar://problem/15530121>)}

We're unsure whether one can test the device environment outside of the app under test. The Xcode's recorder feature suggests so but tests always fail when trying to run them and the documentation isn't clear about it.

Steps to Reproduce:
1.Create an empty project with UI Test support.
2.Add a UITextView with http://www.example.com as text
3.Disable 'editable' flag and enable 'link detection' on the UITextView
4.Record a test that taps on the UITextView and taps on the status bar "back to app" button.
5.Run the test, the aforementioned assertion will occur

Expected Results:
UI Test be able to run outside app under test

Actual Results:
Assertion occurs

Version:
Xcode 7.3.1 and Xcode 8 beta 5

Comments

It's 2019 and I'm still having this issue with Xcode 11. Very disappointing.


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!