Diagnosing Mac OS X Kernel Panics

image

So, pretty much out of nowhere, my trusty old Macbook Pro started shutting down on me. Frequently. Sometimes several times within an hour. Sometimes it would go for hours on end with no trouble. This continued for a couple months of me trying different things: not putting the machine to sleep between uses, switching browsers, uninstalling extensions, I really had no idea when it all started. I seriously considered having the logic board replaced, which would not have been cheap. After hours of combing and reading the message boards and all the info I could find about kernel panics, I finally identified the culprit. Sophos Anti-Virus.

What is a kernel panic?

A kernel panic is an action taken by an operating system upon detecting an internal fatal error from which it cannot safely recover. The Unix kernel maintains internal consistency and runtime correctness with assertions as the fault detection mechanism. The basic assumption is that the hardware and the software should perform correctly and a failure of an assertion results in a panic, i.e. a voluntary halt to all system activity. When a kernel panic occurs in Mac OS X 10.2 through 10.7, the computer displays a multilingual message informing the user that they need to reboot the system. Prior to 10.2, a more traditional Unix-style panic message was displayed; in 10.8, the computer automatically reboots and displays a message after the restart. The format of the message varies from version to version.

A Google search of the topic will reveal many threads claiming that kernel panics are usually hardware problems and that you should take your machine to the Apple store and have the logic board replaced. Believe me, I considered it since I wasn't familiar with the crash reports yet and this problem is one of the most frustrating I've ever come across. Let's face it, who wouldn't be frustrated by what is supposed to be a high-end computer shutting down for no apparent reason several times a day or even several times an hour sometimes? Nobody, that's who.

The crash report

Here is an example of the crash reports I was seeing, yours may be different and I can't say for sure what yours will reveal, but hopefully describing the process I went through will help others to dissect and decode the data in the crash report.

Interval Since Last Panic Report: 53825 sec
Panics Since Last Report: 1
Anonymous UUID: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX

Wed Sep 18 09:26:25 2013
panic(cpu 0 caller 0xffffff7f9ddfcf1a): "GPU Panic: [<None>] 3 3 7f 0 0 0 0 3 : NVRM[0/1:0:0]: Read Error 0x00000100: CFG 0xffffffff 0xffffffff 0xffffffff, BAR0 0xd2000000 0xffffff811d162000 0x0a5480a2, D0, P3/4\n"@/SourceCache/AppleGraphicsControl/AppleGraphicsControl-3.4.5/src/AppleMuxControl/kext/GPUPanic.cpp:127
Backtrace (CPU 0), Frame : Return Address
0xffffff80fe2cadd0 : 0xffffff801bc1d626
0xffffff80fe2cae40 : 0xffffff7f9ddfcf1a
0xffffff80fe2caf10 : 0xffffff7f9c362f1e
0xffffff80fe2cafd0 : 0xffffff7f9c43712d
0xffffff80fe2cb010 : 0xffffff7f9c43718e
0xffffff80fe2cb080 : 0xffffff7f9c6fded0
0xffffff80fe2cb1b0 : 0xffffff7f9c45fa75
0xffffff80fe2cb1d0 : 0xffffff7f9c369d50
0xffffff80fe2cb280 : 0xffffff7f9c3677d0
0xffffff80fe2cb480 : 0xffffff7f9c36910a
0xffffff80fe2cb550 : 0xffffff7f9d3714a8
0xffffff80fe2cb690 : 0xffffff7f9d370f94
0xffffff80fe2cb6a0 : 0xffffff7f9d334348
0xffffff80fe2cb6b0 : 0xffffff7f9d3343bd
0xffffff80fe2cb6c0 : 0xffffff7f9d31543c
0xffffff80fe2cb700 : 0xffffff7f9d317b6c
0xffffff80fe2cb730 : 0xffffff7f9d36b15d
0xffffff80fe2cb7b0 : 0xffffff7f9d3523e0
0xffffff80fe2cb810 : 0xffffff7f9d352cc1
0xffffff80fe2cb860 : 0xffffff7f9d353214
0xffffff80fe2cb8d0 : 0xffffff7f9d3539d3
0xffffff80fe2cb910 : 0xffffff7f9d31ee2f
0xffffff80fe2cba90 : 0xffffff7f9d34ff0f
0xffffff80fe2cbb50 : 0xffffff7f9d31d8c8
0xffffff80fe2cbba0 : 0xffffff801c066e69
0xffffff80fe2cbbc0 : 0xffffff801c068410
0xffffff80fe2cbc20 : 0xffffff801c065e2f
0xffffff80fe2cbd70 : 0xffffff801bc98c01
0xffffff80fe2cbe80 : 0xffffff801bc20b3d
0xffffff80fe2cbeb0 : 0xffffff801bc10448
0xffffff80fe2cbf00 : 0xffffff801bc1961b
0xffffff80fe2cbf70 : 0xffffff801bca6536
0xffffff80fe2cbfb0 : 0xffffff801bcce9e3
Kernel Extensions in backtrace:
com.apple.driver.AppleMuxControl(3.4.5)[49FEF732-D7A3-327B-A7AA-6AC5A6E3DCFF]@0xffffff7f9ddef000->0xffffff7f9de01fff
dependency: com.apple.driver.AppleBacklightExpert(1.0.4)[1D0BB11E-7D71-34CF-ACC3-57DF01CADA08]@0xffffff7f9ddea000
dependency: com.apple.iokit.IOPCIFamily(2.7.3)[1D668879-BEF8-3C58-ABFE-FAC6B3E9A292]@0xffffff7f9c26b000
dependency: com.apple.driver.AppleGraphicsControl(3.4.5)[4A2C8548-7EF1-38A9-8817-E8CB34B8DCA6]@0xffffff7f9ddd6000
dependency: com.apple.iokit.IOACPIFamily(1.4)[A35915E8-C1B0-3C0F-81DF-5515BC9002FC]@0xffffff7f9c1e8000
dependency: com.apple.iokit.IONDRVSupport(2.3.7)[6C8CFC18-75F0-3DEF-86C7-CEB2C1FD6BB1]@0xffffff7f9c2ed000
dependency: com.apple.iokit.IOGraphicsFamily(2.3.7)[990D1A42-DF16-3AB9-ABC1-6A88AC142244]@0xffffff7f9c2aa000
com.apple.NVDAResman(8.1.2)[96AE69DE-8A37-39D0-B2D3-D8446A6AA670]@0xffffff7f9c301000->0xffffff7f9c5a6fff
dependency: com.apple.iokit.IOPCIFamily(2.7.3)[1D668879-BEF8-3C58-ABFE-FAC6B3E9A292]@0xffffff7f9c26b000
dependency: com.apple.iokit.IONDRVSupport(2.3.7)[6C8CFC18-75F0-3DEF-86C7-CEB2C1FD6BB1]@0xffffff7f9c2ed000
dependency: com.apple.iokit.IOGraphicsFamily(2.3.7)[990D1A42-DF16-3AB9-ABC1-6A88AC142244]@0xffffff7f9c2aa000
com.apple.GeForce(8.1.2)[7EC545A4-4B57-32F1-8DC3-C31023AFBDCB]@0xffffff7f9d30b000->0xffffff7f9d3d8fff
dependency: com.apple.NVDAResman(8.1.2)[96AE69DE-8A37-39D0-B2D3-D8446A6AA670]@0xffffff7f9c301000
dependency: com.apple.iokit.IONDRVSupport(2.3.7)[6C8CFC18-75F0-3DEF-86C7-CEB2C1FD6BB1]@0xffffff7f9c2ed000
dependency: com.apple.iokit.IOPCIFamily(2.7.3)[1D668879-BEF8-3C58-ABFE-FAC6B3E9A292]@0xffffff7f9c26b000
dependency: com.apple.iokit.IOGraphicsFamily(2.3.7)[990D1A42-DF16-3AB9-ABC1-6A88AC142244]@0xffffff7f9c2aa000
com.apple.nvidia.nv50hal(8.1.2)[988EAF3A-3318-3787-8A5A-844830FA1522]@0xffffff7f9c5b2000->0xffffff7f9c886fff
dependency: com.apple.NVDAResman(8.1.2)[96AE69DE-8A37-39D0-B2D3-D8446A6AA670]@0xffffff7f9c301000
dependency: com.apple.iokit.IOPCIFamily(2.7.3)[1D668879-BEF8-3C58-ABFE-FAC6B3E9A292]@0xffffff7f9c26b000

BSD process name corresponding to current thread: WindowServer

Mac OS version:
12E55

Kernel version:
Darwin Kernel Version 12.4.0: Wed May 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64
Kernel UUID: 896CB1E3-AB79-3DF1-B595-549DFFDF3D36
Kernel slide: 0x000000001ba00000
Kernel text base: 0xffffff801bc00000
System model name: MacBookPro6,2 (Mac-F22586C8)

System uptime in nanoseconds: 51413455730117
last loaded kext at 19263259633095: com.sophos.kext.sav 8.0.14 (addr 0xffffff7f9df43000, size 24576)
last unloaded kext at 19254333897985: com.sophos.kext.sav 8.0.14 (addr 0xffffff7f9df3b000, size 20480)
loaded kexts:
com.sophos.kext.sav 8.0.14
com.apple.driver.AppleUSBCDC 4.1.23
com.apple.filesystems.smbfs 1.8.4
com.apple.filesystems.msdosfs 1.8.1
com.apple.driver.AppleHWSensor 1.9.5d0
com.apple.iokit.IOBluetoothSerialManager 4.1.4f2
com.apple.filesystems.autofs 3.0
com.apple.driver.AudioAUUC 1.60
com.apple.driver.AGPM 100.12.87
com.apple.driver.AppleTyMCEDriver 1.0.2d2
com.apple.driver.AppleHDAHardwareConfigDriver 2.3.7fc4
com.apple.iokit.IOUserEthernet 1.0.0d1
com.apple.driver.AppleHDA 2.3.7fc4
com.apple.driver.AppleMikeyHIDDriver 122
com.apple.GeForce 8.1.2
com.apple.iokit.IOBluetoothUSBDFU 4.1.4f2
com.apple.driver.AppleSMCLMU 2.0.3d0
com.apple.driver.AppleBacklight 170.2.5
com.apple.driver.AppleUpstreamUserClient 3.5.10
com.apple.iokit.BroadcomBluetoothHCIControllerUSBTransport 4.1.4f2
com.apple.driver.AppleIntelHDGraphics 8.1.2
com.apple.driver.AppleIntelHDGraphicsFB 8.1.2
com.apple.driver.AppleLPC 1.6.0
com.apple.driver.AppleSMCPDRC 1.0.0
com.apple.driver.AppleMikeyDriver 2.3.7fc4
com.apple.nvidia.NVDAStartup 8.1.2
com.apple.driver.AppleMuxControl 3.4.5
com.apple.driver.ACPI_SMC_PlatformPlugin 1.0.0
com.apple.driver.AppleMCCSControl 1.1.11
com.apple.Dont_Steal_Mac_OS_X 7.0.0
com.apple.driver.ApplePolicyControl 3.4.5
com.apple.driver.SMCMotionSensor 3.0.3d1
com.apple.driver.AppleUSBTCButtons 237.1
com.apple.driver.AppleIRController 320.15
com.apple.driver.AppleUSBCardReader 3.1.7
com.apple.driver.AppleUSBTCKeyEventDriver 237.1
com.apple.driver.AppleUSBTCKeyboard 237.1
com.apple.iokit.SCSITaskUserClient 3.5.5
com.apple.driver.AppleFileSystemDriver 3.0.1
com.apple.AppleFSCompression.AppleFSCompressionTypeDataless 1.0.0d1
com.apple.AppleFSCompression.AppleFSCompressionTypeZlib 1.0.0d1
com.apple.BootCache 34
com.apple.driver.XsanFilter 404
com.apple.iokit.IOAHCIBlockStorage 2.3.1
com.apple.driver.AppleUSBHub 5.5.5
com.apple.driver.AirPort.Brcm4331 615.20.17
com.apple.iokit.AppleBCM5701Ethernet 3.6.1b4
com.apple.driver.AppleAHCIPort 2.5.2
com.apple.driver.AppleFWOHCI 4.9.6
com.apple.driver.AppleUSBEHCI 5.5.0
com.apple.driver.AppleUSBUHCI 5.2.5
com.apple.driver.AppleEFINVRAM 1.7
com.apple.driver.AppleSmartBatteryManager 161.0.0
com.apple.driver.AppleRTC 1.5
com.apple.driver.AppleACPIButtons 1.7
com.apple.driver.AppleHPET 1.8
com.apple.driver.AppleSMBIOS 1.9
com.apple.driver.AppleACPIEC 1.7
com.apple.driver.AppleAPIC 1.6
com.apple.driver.AppleIntelCPUPowerManagementClient 196.0.0
com.apple.nke.applicationfirewall 4.0.39
com.apple.security.quarantine 2.1
com.apple.driver.AppleIntelCPUPowerManagement 196.0.0
com.apple.iokit.IOSerialFamily 10.0.6
com.apple.kext.triggers 1.0
com.apple.driver.DspFuncLib 2.3.7fc4
com.apple.iokit.IOAudioFamily 1.8.9fc11
com.apple.kext.OSvKernDSPLib 1.6
com.apple.nvidia.nv50hal 8.1.2
com.apple.iokit.IOSurface 86.0.4
com.apple.NVDAResman 8.1.2
com.apple.iokit.IOFireWireIP 2.2.5
com.apple.iokit.AppleBluetoothHCIControllerUSBTransport 4.1.4f2
com.apple.driver.AppleHDAController 2.3.7fc4
com.apple.iokit.IOHDAFamily 2.3.7fc4
com.apple.driver.AppleSMBusPCI 1.0.11d0
com.apple.driver.AppleBacklightExpert 1.0.4
com.apple.driver.IOPlatformPluginLegacy 1.0.0
com.apple.driver.IOPlatformPluginFamily 5.3.0d51
com.apple.driver.AppleSMBusController 1.0.11d0
com.apple.iokit.IOBluetoothFamily 4.1.4f2
com.apple.driver.AppleGraphicsControl 3.4.5
com.apple.iokit.IONDRVSupport 2.3.7
com.apple.iokit.IOGraphicsFamily 2.3.7
com.apple.driver.AppleSMC 3.1.4d2
com.apple.iokit.IOSCSIBlockCommandsDevice 3.5.5
com.apple.iokit.IOUSBMassStorageClass 3.5.1
com.apple.driver.AppleUSBMultitouch 237.3
com.apple.iokit.IOUSBHIDDriver 5.2.5
com.apple.driver.AppleUSBMergeNub 5.5.5
com.apple.driver.AppleUSBComposite 5.2.5
com.apple.iokit.IOSCSIMultimediaCommandsDevice 3.5.5
com.apple.iokit.IOBDStorageFamily 1.7
com.apple.iokit.IODVDStorageFamily 1.7.1
com.apple.iokit.IOCDStorageFamily 1.7.1
com.apple.iokit.IOAHCISerialATAPI 2.5.1
com.apple.iokit.IOSCSIArchitectureModelFamily 3.5.5
com.apple.iokit.IOUSBUserClient 5.5.5
com.apple.iokit.IO80211Family 530.4
com.apple.iokit.IOEthernetAVBController 1.0.2b1
com.apple.iokit.IONetworkingFamily 3.0
com.apple.iokit.IOAHCIFamily 2.3.1
com.apple.iokit.IOFireWireFamily 4.5.5
com.apple.iokit.IOUSBFamily 5.6.0
com.apple.driver.AppleEFIRuntime 1.7
com.apple.iokit.IOHIDFamily 1.8.1
com.apple.iokit.IOSMBusFamily 1.1
com.apple.security.sandbox 220.3
com.apple.kext.AppleMatch 1.0.0d1
com.apple.security.TMSafetyNet 7
com.apple.driver.DiskImages 345
com.apple.iokit.IOStorageFamily 1.8
com.apple.driver.AppleKeyStore 28.21
com.apple.driver.AppleACPIPlatform 1.7
com.apple.iokit.IOPCIFamily 2.7.3
com.apple.iokit.IOACPIFamily 1.4
com.apple.kec.corecrypto 1.0
Model: MacBookPro6,2, BootROM MBP61.0057.B0F, 2 processors, Intel Core i7, 2.66 GHz, 8 GB, SMC 1.58f17
Graphics: Intel HD Graphics, Intel HD Graphics, Built-In, 288 MB
Graphics: NVIDIA GeForce GT 330M, NVIDIA GeForce GT 330M, PCIe, 512 MB
Memory Module: BANK 0/DIMM0, 4 GB, DDR3, 1067 MHz, 0x80CE, 0x4D34373142353237334248312D4346382020
Memory Module: BANK 1/DIMM0, 4 GB, DDR3, 1067 MHz, 0x80CE, 0x4D34373142353237334248312D4346382020
AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0x93), Broadcom BCM43xx 1.0 (5.106.98.100.17)
Bluetooth: Version 4.1.4f2 12041, 2 service, 18 devices, 1 incoming serial ports
Network Service: Wi-Fi, AirPort, en1
Serial ATA Device: WDC WD7500BPKT-75PK4T0, 750.16 GB
Serial ATA Device: MATSHITADVD-R UJ-898
USB Device: hub_device, 0x0424 (SMSC), 0x2514, 0xfa100000 / 2
USB Device: iPhone, apple_vendor_id, 0x12a0, 0xfa140000 / 6
USB Device: BRCM2070 Hub, 0x0a5c (Broadcom Corp.), 0x4500, 0xfa110000 / 5
USB Device: Bluetooth USB Host Controller, apple_vendor_id, 0x8218, 0xfa113000 / 8
USB Device: Internal Memory Card Reader, apple_vendor_id, 0x8403, 0xfa130000 / 4
USB Device: Apple Internal Keyboard / Trackpad, apple_vendor_id, 0x0236, 0xfa120000 / 3
USB Device: hub_device, 0x0424 (SMSC), 0x2514, 0xfd100000 / 2
USB Device: Built-in iSight, apple_vendor_id, 0x8507, 0xfd110000 / 4
USB Device: IR Receiver, apple_vendor_id, 0x8242, 0xfd120000 / 3

 What you need to know

 The portion that is important to what we are doing here is the part about kexts, particularly if there is a mention of the last loaded kext or the last unloaded kext. If either of those point to a non-Apple function, then chances are you have your culprit. For instance, in our crash report the kexts section read:

System uptime in nanoseconds: 51413455730117
last loaded kext at 19263259633095: com.sophos.kext.sav 8.0.14 (addr 0xffffff7f9df43000, size 24576)
last unloaded kext at 19254333897985: com.sophos.kext.sav 8.0.14 (addr 0xffffff7f9df3b000, size 20480)
loaded kexts:
com.sophos.kext.sav 8.0.14
com.apple.driver.AppleUSBCDC 4.1.23
com.apple.filesystems.smbfs 1.8.4
com.apple.filesystems.msdosfs 1.8.1

 Below this is a bunch more kexts that all start with "com.apple" and we're going to ignore those.

What this section tells us is that there was one non-Apple kext loaded when the error occurred: com.sophos.kext.sav. In addition, the sections above list that sophos kext as being the last one loaded and unloaded, which gives a pretty clear indication that it is the cause of the crash. There may be other non-Apple kextx listed on your report and those are your list of suspects

The solution

Uninstall Sophos. Restart. No more panics.

Your solution may be slightly different. You might need to uninstall or update or even just change the settings on the offending program, but this information should let you know what is most likely causing the panics. I'd recommend first running a few Google searches with the name of the program or kext and keywords "kernel panic" and see what happens. I was easily able to find out that Sophos is a known offender in this regard.

If you uninstall all applications that are loading non-Apple kexts or if the last kext loaded or unloaded points to a com.apple kext, the next step would be either running Apple Hardware Test on your computer if yu can or a clean install of OS X. If you get to the point of a clean install of OS X and are still getting panics or if your machine fails the hardware test, then I foresee a trip to the Apple Store in your future, but for now remember:

dontpanic large