Open Radar

 
22-Jul-2011 10:47 AM Anne McCreery:
'Kernel_2011-07-20-154818_Lab-Administrators-iMac-4.panic' and 'Kernel_2011-07-20-144910_Lab-Administrators-iMac-4.panic' were successfully uploaded

22-Jul-2011 10:47 AM Anne McCreery:
Summary:  When using the auditing system above its normal load, the operating system kernel panics, then kernel panics on boot.

Steps to Reproduce:

1.  Make a backup of audit_control(5) (in /etc/security).

2.  Open the audit_control file and include a 'host:' field after 'flags:' to avoid bug 9817001.

3.  Change 'flags:lo,aa' to 'flags:all' and 'naflags:lo,aa' to 'naflags:all'.  Save the file.

4.  Do 'sudo audit -s' to reread the configuration files.  Check /var/log/system.log to make sure there were no errors reading the file (for example, see bug 9817001).

5.  Heavily load the system.  In our tests, we installed MAMP, then used Apache Bench on a remote machine to do 10 concurrent requests 1000 times.  The local machine usually panicked within 20 seconds.

Note:  Without logging out and back in, all events are filtered through the 'naflags:' setting.  To cause a panic with attributable, user-level processes, log out, then log back in, then open several applications or otherwise load the system.

Expected Results:  We expect the system to slow down in order to handle all the requests and auditing, but do not expect a kernel panic.  Perhaps with 'policy:' set to 'ahlt' instead of 'cnt', we would expect a panic, but it happens regardless of policy.

Actual Results:  The system panics, then panics on reboot.

Regression:  Duplicated results on two separate machines.  Setting the flags to 'all' is of course a worst-case scenario, but the system panics with flags 'fr,fw,lo,aa' as well, it just takes a heavier load and a longer time.

Notes:  We think that the problem has something to do with the audit_q or audit_pre_q filling up or not being allocated properly.  The particular problem area seems to be in xnu_root/bsd/security/audit/audit.c, in the audit_new routine, particularly around line 407.

This bug was found while attempting to configure the OS X audit system for performance benchmarking, as part of a survey of auditing systems on several platforms.  This research is being done by Benjamin Kuperman (advisor), Luke Lovett (student), and Isaac McCreery (student) at Oberlin College Computer Science as part of the Oberlin Summer Research Institute.

22-Jul-2011 12:02 PM Anne McCreery:
Steps to Reproduce (more):

6.  Once the system has panicked, it will panic again on boot.  To fix this, boot in target disk mode or from install media, and replace audit_control from the backup you made in step 1.