Carbon for the Cocoa Guy: Handles

By uliwitness

One of the more confusing aspects of MacOS programming is the Handle. That doesn’t have to be so. I’ll quickly illustrate the history of the Handle and then everything should become clearer. I’ll also include a little memory-management-101 at the beginning.

Memory Fragmentation

When you use malloc() or similar APIs to allocate memory, you face the problem of fragmentation: Imagine you have the following (top) situation in memory:

[Illustration of three consecutive memory blocks, where the one in the middle gets deleted, fragmenting memory]

 

You have three blocks of memory: The blue one, the green one and the red one, each 6 bytes in length. Now, you dispose of the green one by calling free() (bottom). Now there’s a six-byte hole between the two blocks. The total free memory (white blocks) is 108 bytes. Trouble is, since the computer can only allocate contiguous blocks of memory, the largest block you can allocate is 102 bytes.

 

Now, we can’t just move the red block to the left to make more continuous free space available, because our program keeps track of each block by its position (its “address”), and moving it would change that address. The system would have to go through your program and change each occurrence of the moved block’s address, which is simply impossible since only your program knows which parts of its memory are used for what. Not to mention it would cause pauses in execution. So, if you needed 103 bytes, you couldn’t get them, even though we have 108 free bytes in total, more than we’d need.

[Illustration of three pointers pointing to the same memory block]

 

Handles – a modern solution to fragmentation

So, what Apple did is they created the Handle. A Handle is essentially a centralized way of storing pointers to memory blocks, so the system only has to change one centralized pointer when it needs to move a block to make more memory available.

[Illustration of three Handles pointing at a master pointer, which in turn points at the memory block]

 

Each pointer to the actual memory is owned by the system, and kept in a central table of pointers (the “master pointer block”). When you want to allocate memory, you ask the system to do it for you using the NewHandle() function. The system gives you a Handle, which is a pointer to the actual master pointer it owns. You only use this Handle to remember where your memory is. When you want to access the memory, you de-reference the Handle once to get at the pointer, and then use that pointer like any other.

The advantage of this is that all access to memory goes through those central master pointers. When the OS has to move a block of memory, all it has to do is change the master pointer, because all Handles point there. The OS also takes care to bunch up all master pointers at the start of memory, and to re-use old master pointers, so that they can’t fragment memory.

Since we don’t want the OS to move around our master pointer while we are using it, there is an HLock() function that lets you mark a Handle as immovable, and HUnlock() to make it movable again as soon as you are finished. To get rid of a Handle, there is the DisposeHandle() function (So, NewHandle/DisposeHandle can be seen as roughly equivalent to malloc() and free(), though they’re not exchangeable).

History, or Carbon, intervenes

By the time Apple switched to Mac OS X, pretty much everything was a Handle. There were ControlHandles used for buttons, MenuHandles used for menus … Since Mac OS X was very different under the hood, Apple needed to change the way menus and controls worked. Mac OS 9 had a single address space, so it was easy to just hand a menu Handle between applications.

But Mac OS X has protected memory. For security reasons, every application runs in its own segregated area of memory, with very limited access to memory in other applications. Moreover, the Window Server, responsible for drawing windows and the menu bar, is now a separate application. You can’t just pass it a MenuHandle, it has to be something different.

Since Apple didn’t want to break everyone’s code, they changed the name of ControlHandle to ControlRef, but kept the old name for compatibility, even though it isn’t a Handle anymore. Which means you have to be extra careful with xxxHandle data types.

Handles – a solution still used today

Unix offers a slightly improved version of the same approach: Each address a program uses isn’t a real address on the RAM chip, but rather a “virtual” address that gets translated to a physical address in RAM. There is a translation table for translating addresses from virtual to physical. When the OS needs to move a block of memory on the chip, it essentially just changes this translation table (I’m simplifying here). So, your code doesn’t even have to know its memory just moved, doesn’t have to mess with Handles. But that needs a faster, better CPU with a Memory Management Unit, which early Macs with their 680×0 CPUs didn’t have (at least not until the Performas and Quadras).

Since Mac OS X is a descendant of BSD Unix, it inherited this, and Apple uses this new Unix-style mechanism in most new APIs. Also, in Mac OS X, a Handle is never moved. Since, under the hood, the master pointer’s memory block is allocated using malloc(), it’s not necessary anymore. So, if you write new code, use malloc()/free(). If you need to talk with APIs that still use Handles, use NewHandle()/DisposeHandle() and double-de-reference, but there’s no need to call HLock()/HUnlock() anymore. Though it doesn’t hurt either.

Share your thoughts