Test Suite Fails randomly

Originator:pcifani
Number:rdar://15559470 Date Originated:11/29/2013
Status:Open Resolved:
Product:Developer Tools Product Version:
Classification: Reproducible:Sometimes
 
Pierluigi Cifani
29-Nov-2013 01:11 PM

Summary:
NOTE: possible duplicate of rdar://15369540

A fixed set tests against a fixed set of code, sometimes passes, but other times fails with some unknown error like "Unexpected TestCaseDidFinish" or "Unexpected TestSuiteWillFinish". This is reproducible with Xcode 5.0.2 and with Xcode Bots

Steps to Reproduce:
1.- Write tests
2.- Write some code that passes the test
3.- Build & Test multiple times on your local machine
4.- Build & Test multiple times on a remote Xcode bot

Expected Results:
The test suite behaviour should be consistent

Actual Results:
The test suite behaviour is random, sometimes not executing some tests and giving cryptic error messages such as "testFeature encountered an error (Unexpected TestSuiteWillFinish)" or "testFeature encountered an error (Unexpected TestCaseDidFinish)"

Version:
Mac OS X 10.9 (13A603)
Xcode 5.0.2 (5A3005)

Notes:
Included screenshots of the Xcode and Xcode-Bot output

Configuration:
I haven't found a consistent way to reproduce, but it's definitely more likely to happen in slower Macs

Attachments:
'Local Xcode.png', 'Local Xcode Passes.png', 'Remote Xcode.png' and 'Remote Xcode Passes.png' were successfully uploaded.

Comments


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!