Unit tests always run in the same order, but should be randomized
| Originator: | ash | ||
| Number: | rdar://34873492 | Date Originated: | 7/11/2017 |
| Status: | Closed as dupe (11719679) | Resolved: | |
| Product: | Developer Tools | Product Version: | |
| Classification: | Reproducible: |
Unit tests should be able to be run based on a random order, to prevent application state from leaking between tests. Ideally, the tests could be run with a seed that is logged, and can be re-used to generate the same order. This isn’t possible in Xcode but it’s pretty integral to the way that developers in Ruby and JavaScript test their code. For example, consider two tests that test code that reads and writes UserDefaults. If the tests are always run in the same order, it’s possible to get false positives order get different results when running only a single test.
Comments
Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!