XCTest Randomized Execution Order: Rerun with same order

Originator:stephen.gurnett
Number:rdar://46273134 Date Originated:27-November-2018
Status:Open Resolved:
Product:Developer Tools Product Version:Xcode 10.1
Classification: Reproducible:
 
- When randomized execution order is used,
- and this reveals a problem,
- we need a way to keep rerunning tests in the same order, so that we know when we have successfully addressed that problem.

Example from RSpec: http://blog.davidchelimsky.net/blog/2012/01/04/rspec-28-is-released/

Nice-to-have: Even with the same seed, the order may change if tests are added / deleted / renamed. It would be nice if Xcode noted these test names, and warned me if
- I am specifying a seed to use for shuffling, and
- any test names have changed.

Comments


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!