Out of Cheese

Apple recently released Xcode 1.2 with the standard assortment of bug fixes and new features. One new feature is a bit hidden, though, and since I’ve only seen it mentioned elsewhere once, I figure it’s high time I let some other folks know. It’s called libgmalloc, or Guard Malloc if you’re into marketing terms.

libgmalloc is the best tool in existence for finding memory access bugs on Mac OS X. It’ll track down references to uninitialized memory, reading off the end of an array, accesses to freed memory, and so on. There’s a catch, though. When you run an application with libgmalloc, it’ll run significantly slower — perhaps two orders of magnitude slower — and it’ll use far more memory than it otherwise would. If you have a very fast system with tons of memory, you might be able to run various applications with it on a regular basis; if not, you might just test your application every so often.

First, though, you have to install libgmalloc. It’s a bit hidden. On the Xcode 1.2 Tools CD, you’ll find a Preview folder at the root level. Open that up and install the April2004LibgmallocPreview package. (No, I don’t know why there aren’t any spaces in the package name.)

Once it’s installed, running it is simple if you’re comfortable with the command line. Unfortunately, there’s no integration with Xcode yet. To use libgmalloc, you’ll have to set up two environment variables. I’ll use tcsh syntax here because that’s the shell I use; if you use bash, you’ll have to translate.

% setenv DYLD_FORCE_FLAT_NAMESPACE 1
% setenv DYLD_INSERT_LIBRARIES /usr/lib/libgmalloc.B.dylib

Now just run your application:

% ./myApp

and be very, very patient. If your application runs correctly (albeit slowly), congrats! You probably don’t have any memory stompers.

I frequently use libgmalloc from within gdb. That way I can debug my application immediately when it crashes due to an invalid memory access. Here’s how to do that:

% gdb myApp
gdb% set env DYLD_FORCE_FLAT_NAMESPACE 1
gdb% set env DYLD_INSERT_LIBRARIES /usr/lib/libgmalloc.B.dylib
gdb% run

Note that you want to set the environment variables from within gdb, rather than on the command line before you launch gdb. If you launch them before you launch gdb, you’ll be running gdb with libgmalloc, and it’ll also run a hundred times slower.

If you find crashes in Apple’s frameworks while running with libgmalloc, file bug reports! Mention that you were running with libgmalloc and the engineer who receives the bug will hopefully recognize that it’s unlikely that the crash was caused by a memory smasher in the application’s code.

Update: As of Tiger, DYLD_FORCE_FLAT_NAMESPACE is no longer required for libgmalloc. The only variable you have to set to use libgmalloc is DYLD_INSERT_LIBRARIES.