Open Radar

 
Area:
Foundation

Summary:
When fetching data from a server that uses gzip transfer encoding iOS will repeatedly perform a conditional get and not respect the cache. iOS will go out to the network after the initial cache timeout and continually receive a 304. This does not happen when the server does not use compression and returns a non-gzip’d response.

Steps to Reproduce:
In the sample project attached you can see the difference between a request that uses gzip content-encoding and one that doesn’t. The sample project hits two URLs. Both have a max-age for the content set to 30 seconds to make testing easier.

1. Proxy the device’s traffic so you can see the real network requests (I’ve used Charles proxy during testing).
2. Load up the sample app.
3. Leave the ‘Get gzip file’ switch on.
4. Touch fetch and observe that a request was made out to the server (via proxy) and that results are shown in the app, response code was 200.
5. Hit clear and fetch again within 30 seconds.
6. Observe that the request was shown in the app, but that the network was not hit (iOS served from cache). During this initial time the network won’t be hit no matter how many times you fetch.
7. Wait 30 seconds.
8. Hit clear and fetch again and observe that network was hit (via proxy) and that results are shown in the app, response code was 304.
9. Hit fetch again repeatedly and notice that network was hit (via proxy), response code was 304 again.

An example timeline is a follows (with gzip):

1. Fetch file, 200 response
2. Fetch file again within 30 seconds, network not hit, iOS serves from cache
3. After 30 seconds, a 304 is returned, network was hit
4. From now on 304 will be returned and network will always be hit.

An example timeline without gzip:

1. Fetch file, 200 response
2. Fetch file again within 30 seconds, network not hit, iOS serves from cache
3. After 30 seconds, a 304 is returned, network was hit
4. Fetch file again within 30 seconds, network not hit, iOS serves from cache
5. After 30 seconds, another 304 is returned, network was hit, continues from step 4

Expected Results:
After the initial 30 seconds, when a conditional GET is done and a 304 is returned iOS should not hit the network for the cache timeout period for any further network requests. This would match behaviour when the file isn’t gzip’d.

If you repeat the tests above, but turn off the ‘Get gzip file’ switch you will notice that after a 304 iOS will not hit the network for the cache timeout period. A 304 will be returned after each cache timeout period (30 seconds).

Additionally, in the sample project you can change the variable `enableWorkAround` to true in ViewController.swift which will fix the issue when fetching content that is returned with gzip content-encoding. When repeating the steps with the fix enabled you’ll see the log:

response had a header content-length of: 1546
creating new response to cache with header content-length of: 3806

The fix works by using the NSURLSessionDataDelegate method `URLSession:dataTask:willCacheResponse:completionHandler:` and changing the proposed cached response headers to have a content-length header that is equal to the actual data size of the response that will be cached. You can see this fix in TransferEncodingBugURLCacheHelper.swift in the sample app.

Observed Results:
After the initial 30 second max-age iOS will continue to hit the network each time the content is fetched. iOS seems to ignore the cache.

Version:
iOS 10.3 (14E277), also is reproducible on iOS 9.3

Notes:
In our app the real URLs max-age is 10 minutes. I’ve used these sample URLs to make it easier to reproduce.

Configuration:
To enable a fix for this bug you can change the variable `enableWorkAround` to true in ViewController.swift which will fix the issue when fetching content that is returned with gzip content-encoding.

Sample project:
https://github.com/gspiers/radar_32270378


Update:

This seems to be fixed in iOS 11 Beta 5, the bug is still open in Radar so will close this when iOS 11 ships.