XCTest: no obvious way to verify outputs from unknown # of async events

Originator:DaNmarner
Number:rdar://29780800 Date Originated:21-Dec-2016 05:20 PM
Status:Open Resolved:
Product:Developer Tools Product Version:Xcode 8.2
Classification:Feature (New) Reproducible:Always
 
Summary:
It’s hard to collect output from unknown number of async events and verify it has expected outcome with XCTestExpectation.

Steps to Reproduce:
As an example, imagine a notification system what gives its callback two payloads: the new one and the previous.

notification.subscribe { new, old in
  newValue = new
  oldValue = old
}
notification.trigger(with: aValue)
notification.trigger(with: aValue)


We want to verify both `oldValue` and `newValue`, but it’s unclear when to call `fulfill()` on expectation in the call back block, since calling it multiple times is prohibited.

Another scenario is this: we expect a certain value from an asynchronous process, but there are other events that needs to run after the expected value is captured, we don’t know for sure when all async output will become available. Calling `fulfill` will terminate the tests, therefore this scenario is unsupported.

Notes:
It’s possible to do it with a timer. But the ultimate feature to have is something similar to the `toEventually` operation in Quick/Nimble. As mentioned in 
rdar://29780337.

Comments


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!