Open Radar

 
Summary:

The latest guidelines mention that only user-created data can be placed in the Documents folder on iOS. This is being interpreted by App Review to mean everything other than Library/Caches and /tmp. The reason is that data placed outside those two folders will be backed up by iTunes or iCloud.

The guidance is that we should place data we can re-download into Caches or tmp. For Kobo, this includes the user's entire library, which may number in the hundreds of megabytes (mine, for example, is about 220MB). Now, obviously this is going to be problematic from a backup perspective— we agree on this issue. However, by placing the data into the Caches folder we run into an issue where iOS deletes the user's library from underneath us when it wants to free up some space.

The problem is that this is not an uncommon occurrence for many of our users. The 'fill all available space with Music' option in iTunes is rather popular, and results in the deletion of a user's library when they next fire up the device. Sadly, this often occurs as they are beginning their morning or evening commute, at which time they either have no internet access, or only have cellular WLAN access— which isn't conducive to a hefty amount of data transfer.

Note that Apple's iBooks does not suffer from this problem— it gets a free pass, as it is able to store its data inside the Media partition (in /private/var/mobile/Media/Books). This can be considered an unfair competitive advantage, as its competitors are forced to place data into Library/Caches or /tmp, where it can be deleted without their knowledge.

Ideally, we should be able to place such things in a location which cannot be arbitrarily deleted yet does not increase the backup load. For example, in something like a 'Library/LocalData' folder, which is excluded from backups. Alternatively, we should be able to continue using 'Library/Application Support', which is the standard place to store such things on OS X, and we should be able to tag the item as 'not backed up' in a manner similar to the CoreServices backup APIs on OS X.

One step further would be to notify the application that the system wants to reclaim space, and to launch the app in a background mode (at low priority, etc) so it can free up some space itself. That way, the application developer can make choices that would cause the least inconvenience to the user. For instance, the Kobo app might delete all downloaded books with the exception of the user's current read.

Note that we tried doing this in our most recent release, but we were rejected for keeping the currently-used book outside the Caches folder.

This is close to my heart, since without this option I am one of those people whose entire library is deleted every time I sync overnight with iTunes. I have to log into the app every morning and re-download my library after iTunes re-syncs my music via WiFi each night. At 250MB a pop, that's a lot of data I'm getting through there.

Steps to Reproduce:

Load an app such as Kobo onto the device. Also load iBooks for comparison.
Load a number of books into both applications. I have about 50 full-length books in my account, making about 250MB of application data for the compressed epub files and other cache data.
Sync with iTunes, ensuring that iTunes completely fills the device.
Re-open iBooks. Note that all books are still present.
Re-open Kobo. Note that no books will open (or, if you're using Kobo v5.0 — not yet approved — you should at least be allowed to see a warning that the system deleted your library and you have to re-download it).

Expected Results:

I loaded lots of books into both iBooks and Kobo. I expect them to damned well stay there until I, the user, choose to delete them.

Actual Results:

iBooks content is safe. Kobo content is deleted, resulting in books which don't open. Since the application isn't informed of the change, the 4.x releases of the Kobo app won't even realize that the files have gone missing until you try to interact with each one. 5.x will detect this automatically and warn the user, provided the app review team doesn't reject the app for doing so (and to be fair, they *did* reject us for trying to explain why our bookstore wasn't available any more).

Regression:

I believe this auto-cleaning only happens on iOS 5, although I had expected it to take place at any time on all versions of iOS, which is why I only put non-essential data into the Caches folder.

Notes:

It goes without saying that I'm fairly certain that the App Review team won't allow us to mention Apple's or the operating system's role in the deletion of users' library contents. This means that our users will see 'Kobo' deleting their stuff while iBooks does not, and lambast us publicly. After receiving a ton of bad press when we were made to remove our storefront (along with all mention of our website, our company name — because it  matched our website address — and the ability for new users to use the app at all) we are understandably reticent to let this slide when the solution would seem so simple.