mdimporter uses all memory, causes computer to become unresponsive when importing large email messages

Originator:codesujal
Number:rdar://15695276 Date Originated:12/18/2013
Status:Open Resolved:
Product:Mail.app Product Version:7.1 (1827)
Classification:Crash/Hang Reproducible:Always
 
Summary:
I've found a reproducible situation that was causing my computer to freeze completely from what looks like runaway memory consumption by the system.

The machine, a 2010 iMac, is running Mavericks Server, but I've been able to reproduce, though less consistently, on a Retina MBP running Mavericks (the MBP has 16GB of RAM, while the iMac has 8GB)

After debugging, I was able to see the following pattern consistently:

- Leave Time Machine or Spotlight running.
- After some time, memory pressure graph in Activity Monitor goes red, and Swap starts growing larger and larger. 
- largest process is kernel_task, ranging around ~4.8GB per Activity Monitor. Several mdworker processes show hundreds of MB RAM used, which is unusual AFAICT.
- After a few minutes (5-10), swap will be 10s of GB, and the computer will just freeze.

After doing a bunch of debugging with opensnoop and Activity Monitor, determine that largest mdworker processes were processing .emlx and .partial.emlx files at the time of the freeze.

Narrowed this down to very large email messages - the culprits were around 30MB on disk, but I was able to reproduce with emails down to about 26-28MB or so.

One email is attached.

I can reproduce the runaway memory condition by running

mdimport -d4 /path/to/email/file

It doesn't always crash, sometimes it goes crazy with RAM usage, but then recovers. On the iMac, it usually freezes. On the 16GB RAM MBP, i've only had one or two freezes, but do notice the slowness when the RAM usage goes crazy.

Please let me know if you need additional information. I've attached a sample email file that caused the behavior on my iMac.

Steps to Reproduce:
1. Put the email file below on disk where spotlight will access it.
2. run mdimport -d4 /path/to/emlx/file
3. Watch RAM usage go crazy, possibly crash system.

Expected Results:
Shouldn't crash system. :)

Actual Results:
Runaway RAM usage and system freeze.

Version:
Mail 7.1 (1827) & OS X 10.9.1 13B42

Notes:
I documented more of my debugging process here: http://fatmixx.com/2013/12/10/need-mavericks-troubleshooting-ideas/

Configuration:
My iMac running Mavericks 10.9.1 with Server installed.

 
iMac.spx - configuration file has been successfully uploaded 
iMac.spx - configuration file has been successfully uploaded

Attachments:

Comments


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!