Timeline

View-based NSTableViews: Row 1 should be in the valid visible section

With Mac OS X 10.7 “Lion”, Apple added a second “content mode” to NSTableView: View-based table views. This allows you to simply create your list items as separate views that get reused and reshuffled to simulate scrolling, instead of using the classic approach of “rubber stamping” a more lightweight NSCell repeatedly into the window. If you’ve programmed for iOS before, this is essentially like UITableView works, just for the Mac.

While this works fairly well most of the time, as of this writing it has one severe bug: Occasionally, a view-based NSTableView‘s view hierarchy gets corrupted in some odd way, causing it to hit an assertion and throw an exception. The main message you see in that case is:

Row 1 should be in the valid visible section.

I don’t know what exactly happens, but it happens if you call -reloadData on your table view too early. Now “too early” may sound a little weird, as cell-based NSTableViews work just fine, but for example the view-based content mode doesn’t like getting a -reloadData from the -awakeFromNib method of the controller that loaded it with its NIB/XIB file.

Doing this early on makes sure that your table view breaks, even if it still seems fine afterwards as long as it is empty. As soon as you add items, however, it croaks. Also, sometimes you don’t need to call the method directly. Just get someone else to call it for you, e.g. by setting the automaticallyPreparesContent property on an NSArrayController.

Thanks to:
brettper, who filed a reproducible case in Apple’s bugreporter and posted it on OpenRadar.
Jacob Gorban, who made the connection to the NSArrayController issue.

A proposal: Categories for C++

Categories?

One of the most useful features I’ve used in object-oriented programming languages is the “category”. A category is a way of adding methods to a class dynamically. Take, for example Objective C’s “NSAttributedString” class. As defined in the Foundation framework, it is simply a string where you can attach key-value pairs to a range of it.

Only a category in the GUI framework AppKit actually defines the methods and constants that define what key to use for “bold”, and let you draw such a string to the screen. Yet in everyday use with AppKit, these two disparate parts feel like one homogeneous part. And, more importantly, anyone who wants to write a text-processing command line tool can use NSAttributedString without having to drag in all that unneeded drawing code.

How C++ objects work

But C++ does not have this feature. So let’s come up with an implementation of this concept that a compiler vendor could implement, and that stays true to the core of C++, mainly it’s compile-time determination of as much as possible. But of course we want to be able to add a category to a system class or a class in another module, and want to define virtual methods in a category, and override them in a category on a subclass. So, of course we want to end up at something that looks like this in C++:

category BarSupport : MyObject // extend class MyObject with some methods.
{
    void doBar( MyObject baz );
};

and can be called just like any other method on MyObject, e.g.

MyObject foo, dodo;
foo.doBar( dodo );

If we didn’t have to support virtual methods, things would be trivial. A non-virtual method call like

MyObject foo;
foo.doBar(baz);

compiles to something like

MyObject_doBar( foo, baz );

Where the this pointer is simply passed in as the first parameter before the first parameter you defined. So all we’d have to do is tell the parser about these new methods, compile the functions that implement them, and call them.

But virtual methods work differently. First, there is a virtual function table, something like this:

struct MyObjectVTable
{
    void (*doFoo)( struct MyObject* this ); // defined by user as doFoo(void).
};

And whenever you define a new class, what it actually does is add a hidden instance variable at the start:

struct MyObject
{
    struct MyObjectVTable *vtable;
    int                   firstInstanceVariable;
};

And it declares a global variable containing the vtable once, for all objects created with this class, and stashes it in each new object’s vtable instance variable:

struct MyObjectVTable gMyObjectSharedVTable = { MyObject_doFoo };

...

// Equivalent code to MyObject* foo = new MyObject :
struct MyObject* foo = malloc(sizeof(struct MyObject));
foo->vtable = &gMyObjectSharedVTable;
MyObject_Constructor( foo );

Here, MyObject_doFoo() is a function just like our

MyObject_doBar()

above. But when a virtual method is called, it is done slightly differently:

foo.vtable->doFoo( &foo );

This means that a subclass that wants to override doFoo() can provide its own gMySubclassSharedVTable that is also a struct MyObjectVTable, but contains a pointer to MySubclass_doFoo instead, and thus overrides the behaviour of doFoo() for all MySubclass objects. And if it wants to define additional virtual methods in its subclass, it can simply declare a struct MySubclassVTable that starts with the same fields in the same order as struct MyObjectVTable, and contains the additional methods after that. That way, anyone who expects an object for the base class doesn’t even have an inkling that there is more stuff after the methods it knows.

Applying that to Categories

Our categories also want to add methods, but the problem is we can’t just add new fields to the struct. The system classes like std::string have already been compiled, and we can’t just change their code to add these ivars. But what we can do is declare a parallel class hierarchy for our categories. So, first we declare a struct for our category:

struct MyObject_BarSupportVTable
{
    void (*doBar)( struct MyObject* this, struct MyObject* baz );
};

Then we extend the vtable to list categories:

struct CategoryEntry
{
    MyObject_BarSupportVTable* catVTable; // Simplified, each category's vtable is really a different struct.
};

struct MyObjectVTable
{
    struct CategoryEntry *cattable;
    void (*doFoo)( struct MyObject* this ); // defined by user as doFoo(void).
};

When a class is first built, the cattable array is empty, but as soon as someone declares a category, it gets added to that list. We just add some initialization code at the start of main() that mallocs the list. But how do we look up the vtable for a category? We could store its name, but then we’d have to loop over this list on each call and compare category names until we find it. How can we make that more efficient?

Simple: When we add the category to the class, we remember the index into the array where we added this category in the list, cache it in a global variable int gMyObject_BarSupport_Index; and then we can call a virtual method in a category like:

foo.vtable->cattable[gMyObject_BarSupport_Index].catVTable->doBar( &foo, baz );

Again, a subclass-category that overrides this method can just provide its own doBar method in the struct MyObject_BarSupportVTable.

The cost of categories

Of course this means that, just like with C++ virtual methods, and a bit more so, you pay a price for calling a virtual method in a category. You also pay with a little bit of overhead at startup, when the categories are added to the base class vtable. And just like with C++ virtual methods, to override a category method in a subclass, you have to know that the category exists on the base class. And if your program uses threads and you load a dynamic library that contains a category on a system class, bad things could happen while the category lists change under your running code’s rear.

Another gotcha with this approach is that the category list has to be built from the base class to the subclasses, because once a subclass locks down an index in the table for its first category vtable, you can’t add another category to the base class (it would have the index of the subclass’s category). But even that can be fixed:

We can just give every subclass its own categories table. So it only deals with the base class table when it wants to call a method inherited from the base class. E.g.:

struct MySubclassVTable
{
    struct CategoryEntry *cattable;
    void (*doFoo)( struct MyObject* this ); // defined by user as doFoo(void).
    struct CategoryEntry2 *cattable2;
    void (*doBar)( struct MyObject* this, struct MyObject baz ); // defined by user as doBar( MyObject baz ).
};

And now, calling a method in a category specific to the subclass simply goes through the cattable2.

Thoughts, suggestions, additional runtime geekery?

How to build a good restaurant web site

The typical restaurant web site, I’ve found, is completely useless and a waste of money. Here’s a short list why:

  • Most of them are 100% Flash. Nobody who owns a smart phone can view them. At all. So if I’m on the road and want to know whether your restaurant is open, I can’t see that, just because you wanted a photo slideshow with crossfades.
     
  • Most of them are missing the opening hours and/or the address. Those that have them often hide them in lots of prose. Someone on the road with their phone will want to know that information first.
     
  • Most of them are missing the menus. While some of them have the permanent menu, particularly the daily lunch deals or weekly changing menus are why a prospective customer might come back to your web site.
     

Of course, everyone can moan and complain, so here’s my short and sweet summary of how to make a good web site for a restaurant:

  • Put the following on your front page: Your address (including the city and country, this is the internet, after all!), your opening hours, and a tag line like “Greek taverna” or “Italian kitchen” or “exclusive 4-course dining in separés” or something else that helps a first-time visitor immediately get an idea of your restaurant.
     
    And no, your address as the “legally responsible party” on your web site’s imprint page doesn’t count. That could be an office building for a restaurant chain. Make sure it’s clear where to go. Put a small picture of your front entrance on there so they recognize it.
     
  • Don’t use Flash. People on cell phones can’t see Flash, they just get a lego brick icon and that’s it.
     
    If someone is in your general area and wants to know where to go, they will call up your site on their smart phone to check the opening hours. Make it easy for them. You’re wasting money if half your interested customers can’t see your site.
     
  • Put your daily menu and specials on the site. This is easier than it sounds. You don’t have to pay a web designer every time. Pay them to make you one editable page where you can just log in with a password and edit the text from any browser. You probably already type up the daily menu and print it every day. Just copy it over there, click “Save” and anyone on the internet (potential customers sitting at work thinking where to go for lunch together, for instance) can immediately see what you have to offer.
     
    Your permanent menu is nice, but people who’ve been at your place a couple times probably have a general idea what’s on it already. The specials change daily or weekly, everyone has to look those up.
     
  • Bonus points: Include a phone number (or even better, a web form) where people can make reservations. Ideally they’d be hooked up to your reservation system and immediately give feedback. Otherwise, make sure you check your e-mail often and confirm reservations in a timely manner.
     
    If you want to provide prose or an image gallery, put them on extra pages, so cell phone visitors don’t have to download all of that over a mobile connection.
     
    And finally: Pay for a professional web designer and professional photographer. It will show in the end result.

That’s my short list of how to make a good, useful restaurant web site. I hope it will help restaurant owners get the right thing from their web designers.

Building a custom NSButton

In the previous article, I illustrated how one can theme NSTableView. Another control that commonly needs to be customized in themed Mac applications is NSButton. Since NSButton is a cell-based NSControl subclass, we don’t actually need to subclass it. Rather, we need to subclass NSButtonCell.

If you are using a XIB file, you can then just select the NSButton‘s NSButtonCell by clicking the button a couple of times, or by uncollapsing the button in the outline view and selecting the cell there, and setting its class to the name of our subclass.

In most cases, it is sufficient to override - (void)drawBezelWithFrame:(NSRect)frame inView:(NSView*)controlView and draw your custom button body there. NSButtonCell will take care of drawing the title, image etc.

The biggest difficulty in that case is to find out when and how to highlight your button. There are two ways a button can be highlighted, which are most obvious in a checkbox button: The actual highlight when a button is pressed is indicated by the -(BOOL) isHighlighted property.

In addition to that, a checkbox can be selected or unselected, which is indicated by its -(NSInteger) state property being set to NSOnState, NSOffState or NSMixedState. However, while every button toggles its state when clicked, not all buttons draw differently when their state is not NSOffState To find out whether you should look at the button’s state, query the -(NSInteger) showsStateBy bit mask to check whether the NSChangeGrayCell bit is set. Your button is only supposed to reflect its state if this bit is set.

If your button shape and position differs from the standard size and shape, you will also want to override - (NSUInteger)hitTestForEvent:(NSEvent *)event inRect:(NSRect)cellFrame ofView:(NSView *)controlView to return NSCellHitContentArea | NSCellHitTrackableArea so the user doesn’t click beside your button or on its shadow to drag the window and accidentally triggers its action, or worse, click on the button to have nothing happen because the standard system button doesn’t extend as far as your custom drawing.

Usually that is all you need. You will get a custom look, but the system will provide all the correct tracking behaviour and implement accessibility actions etc. for you, swap out the title and image for the alternate title and image as needed etc.

However, sometimes you need to also adjust the title text color, e.g. because you are implementing a dark button, or because you want to reflect the button’s selected state through its text color being different from the highlighted variant. To do that, you override - (NSRect)drawTitle:(NSAttributedString*)title withFrame:(NSRect)frame inView:(NSView*)controlView. Note that the frame that is passed to this is usually already correct. Just draw your text at the origin of this rect in the font set on the cell, and you should be fine. And return the actual rectangle in which your drawing ended up.

If the standard button text positioning or image positioning doesn’t match your button’s design, there are - (NSRect)titleRectForBounds:(NSRect)theRect and - (NSRect)imageRectForBounds:(NSRect)theRect for you to override.

Finally, if you plan to create your buttons in code instead of thawing them from a XIB file, you will also have to create a subclass of NSButton and call +(void) setCellClass: (Class)inClass in its +(void) load method to tell it what cell class to instantiate.

Creativity Finds a Way

Uli's xDraw XCMD screenshot

Great observations

There is currently a nice little discussion on HyperCard going on in the comments on Stanislav Datskovskiy’s article Why HyperCard had to Die:

The article looks at the right facts, but I think draws the wrong conclusions: Yes, HyperCard was an echo of an era where a computer was a complex machine, and its owners were tinkerers who needed to customize it before it became useful. Yes, when Steve Jobs came back, he killed a lot of projects. And the Steve Jobs biography mentions that he doesn’t like other people screwing around with his designs.

But I do not think this automatically leads to the conclusion that Apple is on a grand crusade to remove the users’ control over their computers. Nor does it mean what many of the commenters say, that Apple is trying to dumb down programs and that programmers are underestimating their users.

How people work

Every programmer knows how important a coherent whole is: If a button appears in the wrong context, it will easily (and unintentionally) trick the user into thinking it does the opposite of what it really does. You can add paragraphs over paragraphs of text telling your users the opposite and they will not read it.

This is not because users are stupid, but because users “scan”. Screens are complex, and full of data. For the user to find something without spending hours of their life on it, they tend to quickly slide their eyes across the page, looking for words that come from the same category as the thing they are trying to do next.

This is a human efficiency optimization. It is a good thing. If we didn’t have this mechanism, we’d probably all be autistic, and incapable of coping with the world. Once a word is found, the user starts reading a little bit around it to verify the impression that this is what they want, and then they click the button.

It seems trivial to engineer a program for that, but it’s easy to overlook that a computer is not a single application at a time. There are other things happening on the screen, there may be other windows open. There may be system alerts popping up. Even if they are marked with each application’s icon or name, chances are that most users are too busy getting actual work done to memorize application names and icons. They won’t be able to distinguish what is your application, what is another.

Similar with haxies. Any halfway successful programmer probably has a story of how they tried to track down a crash or oddity a user encountered in their program that was actually caused by a plug-in or haxie that injects itself into every application to modify some behaviour system-wide. And once they are installed, even I occasionally forgot I had them installed. Or didn’t expect it to have an effect; Why should a tool that watches when my cursor hits the edge of my screen and then remote-controls the cursor on another computer as if it was an attached screen cause the menu bar to just randomly not show up when switching between applications?

Software is complex. Designing reliable, usable software is complex. In a comment, Stanislav had a great analogy for this (in response to someone’s pipe dream that one would just have to use HTML, and the technical stuff was all already done, you just had to add the human touch):

All the pieces of the world’s greatest statue are sitting inside a granite mountain. Somebody just has to come and chip away all the extra granite, adding the human touch. The technical problems are all virtually solved!

Software is hard. I don’t say this because it makes me sound cooler when I say I’m a programmer, but because you’re not just building a thing. You are building behaviours. HyperCard was notorious for being the tool for the creation of a number of the ugliest, least Mac-like programs ever released on the Mac. Because even with the best camera, your movie is only as good as the camera man.

So was Steve Jobs happy to get rid of HyperCard and stop people from screwing with his design? Probably. Was he forced to let it linger instead of killing it outright because he didn’t want to lose the educational market? I can’t disprove it. But Steve Jobs was also known to be an idealist. He genuinely thought his work would improve the world. What would he gain by making everyone dumb and uncreative?

Why assume malice when Occam’s Razor is a much better explanation?

You can’t hold a good idea down

When the Mac was originally released, it was intended as a machine for everyone. To bring computers to the masses. Almost from day one, the goal of Apple Computer, Inc. has been to drop the darned “Computer” from their name. Compared to the mainframes of the time, the Apple ][ that started the home computing revolution already was a “dumbing down” of computers.

Was this the end of the world? Should we have stayed in the trees? Will people become un-creative? Look around on the net. There are people out there who have no programming skills, who dig around in the games they bought and modify them, create their own levels, use existing game engines to create a game about their favorite book or TV show. Heck, there are people out there who create a 3D game engine in Excel.

If there is one thing we can learn, it is that Creativity Finds a Way.

HyperCard was designed in the late 1980s, for hardware of the time, for what very smart people thought would be the future at the time. Being creative with a computer, at the time, meant writing code. So they gave us a better programming language. Ways to click on a “Link to…” button to create the code to change pages. Not unlike Henry Ford’s competitors would have built you a better horse, but not a car.

Yes, I am saying that the developers of HyperCard didn’t quite anticipate the future correctly. They didn’t anticipate the internet, for example. That’s not a shame. It was ’87 back then. I didn’t get what the internet would be good for in ’91. I probably wouldn’t even have managed to invent a better horse. But anyway, all I am saying is that HyperCard’s creators didn’t know some things we know now, and probably made some compromises that wouldn’t make sense now.

The world has changed: This is 2011! All our programs do so much more. You can create 3D graphs in Excel, colorful drawings and animations in Keynote, and upload it all to the web with Sandvox. So many tools are available for such low prices. Why would you bother with a low-level, rudimentary tool like HyperCard when all you want to do is a movie with some branching?

A new tool for a new world

After all that, it might surprise you that I still agree with everyone in the comments who says that we need a new HyperCard for the 2010s. However, I do not agree that any of the examples the commenters mentioned (or even HyperCard as it shipped) are this program. Yes, Xcode and the NeXT-descended dev tools, and VB and others use the Rapid Application Development drag-and-drop manipulation to lay out your UI. But guess what? So does Pages.

Yes, you can use Ruby and Python and Smalltalk to branch between different choices. Or you could just use links to move between web pages built using Sandvox.

Yes, you can build real, runnable applications from your work with Java or AppleScript. But why would anyone want to build an application? Movies can be uploaded to YouTube, web sites can be built with WordPress, and I don’t have to transfer huge files to users. I just send my friends the link, and they know what to do. There’s no installer.

Our computing world has become so much richer, so much easier, that it is more efficient and actually smarter to just create your stuff with those tools that we old HyperCarders see as dumb. They can stand on the shoulders of giants, and spend their time creating the best possible gameplay instead of coding yet another 3D renderer. That is why HyperCard 2.4 just won’t cut it, or as David Stevens commented on that very same article:

most people get on a train to go somewhere, not because they really want to lay track, which explains the shortage of track laying machines in yard sales, and the demise of HyperCard.

The new HyperCard won’t be like HyperCard. Maybe the web is enough. Maybe it will just be a good “web editor”, like it used to be included in every copy of Netscape back in the day.

Or maybe, it will just be a niche product aimed at people who find that they want to do more than their tools let them do. This will not be the typical movie-maker, podcaster or writer. Like the directors, radio hosts or journalists in the generations before them, those will specialize. They will be exceptional at directing, making a show or researching the truth. But they will not care how the camera transports the film, they won’t care how their voice is really broadcast as radio waves and re-assembled in the receiver, nor how to build a printing press.

The people a new HyperCard is aimed at will be a person like you, who saw HyperCard, and at some point stood up and said: This is not enough. I want to create more. And then maybe went out and bought CompileIt!, which let her use the system APIs from the comfort of her HyperCard stack, only needing to use the scary memory management stuff when absolutely necessary. And then went and bought MPW, or THINK C, or CodeWarrior, or Xcode.

A real programmer doesn’t program because she wants to use HyperCard. A real programmer programs because she wants to. Because she just has to. A real programmer doesn’t limit herself to that one language and IDE she learned once so she never has to learn anything else. A real programmer learns new programming languages because each language teaches her a new way of looking at or solving a problem. A real programmer has been programming since she opened PowerPoint for the first time. She will find a way.

It’s been like that back in the days of HyperCard. Why shouldn’t it be like that again?

Themeing NSTableView

Themed tstableviewWhile most Mac applications rely on the standard controls provided by MacOS X to do their work, most application developers like to follow Apple’s lead and apply their own textures, colors and fonts. And why not? Screens today have a much higher resolution and aren’t limited to basic B/W shapes. Tinting or texturing a user interface — when done with restraint and taste — helps the user find your application’s windows on multiple-window desktops more easily, and can increase recognizability of your application for other purposes.

But how does one correctly theme user interface controls with minimal interference to Apple-provided default behaviour? After all, we want our application to still look and, more importantly, behave in recognizable and familiar ways to the user. We just want it to look like a different material, or match the application’s logo and packaging better.

Let’s start with a table view. While an NSCollectionView can be easily themed by modifying various views’ color settings in Interface Builder, the best you can do for an NSTableView or its subclass NSOutlineView is set one single solid background color.

We need to write code, then. There is enough documentation for the old NSCell-based table views out there that describe how to override - (void)drawRow:(NSInteger)row clipRect:(NSRect)clipRect to theme those. So I’ll restrict myself to themeing the view-based table view variant. Most of the cells can easily be edited, but what if you want custom alternating row colors?

Luckily, NSTableRowView has a backgroundColor property that you can change. So all you need to do is make sure that you provide a - (void)tableView:(NSTableView *)tableView didAddRowView:(NSTableRowView *)rowView forRow:(NSInteger)row method in your delegate that uses the % operator to grab the corresponding color from an array of alternating row colors and sets the appropriate one as each rowView’s backgroundColor.

However, this only fixes the cells. The area below the rows, or an empty table, will still have whatever standard alternating row colors the system uses. You will have to subclass NSTableView and override - (void)drawBackgroundInClipRect:(NSRect)clipRect to fill the area below the rows.

Call -rowViewAtRow:(NSInteger)row makeIfNecessary: YES to get the last row’s NSMaxY(), if it is inside the bounds, draw alternating rectangles with a height of -(CGFloat) rowHeight underneath it until the row rect is fully out of view (start their row number at -(NSInteger) numberOfRows).

It is unfortunate that there seems to be no public equivalent to tableView:didAddRowView:forRow: in NSTableView itself. That way, one could create one subclass that takes care of the whole appearance consistently, instead having to add code for that to each delegate.

Common Cocoa Coding Patterns: UITableView row index enum

One of the things pretty much every iPhone application has is a table view containing a pre-defined list of main menu items that lead to deeper sections. Many people just hard-code this list and the item indexes, only to later find out that they need to display some rows conditionally. The code becomes a mess of conditional statements duplicated all over the place to adjust the indexes in the case that a certain item does not apply.

But there is an easier way. The first thing you do, is create an enum with the item indexes:

enum
{
    kMainTableRowInbox = 0,
    kMainTableRowTrash,
    kMainTableRowSpam,
    kMainTableNumberOfRows
};

Whenever you would use the indexes, e.g. in your table view data source and delegate methods, use these constants instead. You have the same code as before, but if you want to add/insert a new item, you can just add the constant for it in the right spot, and the rest of your code magically updates itself.

If you have one source file that implements the main list for several similar products, you can even use the preprocessor to remove items that are only needed for one device. However, what do you do if you dynamically need to add/remove items at runtime? E.g. if your application can talk to two kinds of servers, and one has a kMainTableRowTrash, but the other kind doesn’t? Simple: You create a look-up-table:

NSUInteger          sRowLookUpTable[kMainTableNumberOfRows] = { 0 };
static NSUInteger   sRowLookUpTableSize = 0;

-(void) rebuildRowLookUpTable
{
    NSUInteger  actualRowNumber = 0;
    for( NSUInteger virtualRowNumber = 0; virtualRowNumber < kMainTableNumberOfRows; virtualRowNumber++ )
    {
        if( virtualRowNumber == kMainTableRowTrash && ![self haveTrash] )
            continue;
        sRowLookUpTable[actualRowNumber] = virtualRowNumber;
        actualRowNumber++;
    }

    sRowLookUpTableSize = actualRowNumber;
}

Call this in your init method(s), or whenever some state changes that may affect which rows should be shown. Change your table view delegate methods to use sRowLookUpTableSize instead of kMainTableNumberOfRows for the number of rows. And whenever you look at an index to compare it to one of the row index constants, do

sRowLookUpTable[indexPath.row]

instead of just

indexPath.row

to translate the real row index into one of the constants that indicate what content you want displayed. This way, most of your code never needs to know in which order which items are actually shown in your table. It just needs to be able to display all kinds of rows, and only -rebuildRowLookUpTable needs to actually know which rows are hidden or visible.

Also, the constant names make it obvious what row a bit of code is dealing with. If, two years after you wrote an app on contract, the customer comes back and wants it updated for MacBook Airs with iOS, and remove the settings category, you can just search for a kMainTableRowSettings constant in the text editor and get rid of all that code. Or just move the constant below the kMainTableNumberOfRows constant. All the code is still there, compiles, but the index will never be added to the table.

Dennis Ritchie Deceased

Apparently, a few days ago, Dennis Ritchie, the “R” in “K&R”, co-creator of the C programming language has died.

Thank you for laying the groundwork for our profession, Mr. Ritchie.

Thank you, Steve.

Thank you, Steve.

Screenshot of Apple's web site on the day of Steve Jobs's death

We’ll take it from here.

The Sandbox, Pro and Contra

Blog sandbox

Sandbox?

With Lion, Apple has introduced the “Sandbox”. Essentially, it is a way to un-break the unix permission model for the internet age. In ye olde days, user accounts and permissions were designed to prevent one user on a big mainframe from screwing with the files of another working on the same mainframe.

But in this internet age, our user account runs a lot of code that doesn’t come from us: Scripts from web sites, applications a distant acquaintance e-mailed us … While we should be careful to not run untrusted software, the matter of the fact is that we often have no choice, and when we do, we might not have enough information to make an educated decision.

When we run a screen saver, we run it with certain expectations: It should save our screen, not access our address book. It is time that these expectations get formalized in code and property lists, so the computer can enforce them for us. That way, even beginners can be protected.

How does it work?

In short, your application gets locked out of a number of “sensitive” areas. That includes the file system (except for a few places like your Preferences file and your Application Support folder), and also inter-application communication (e.g. AppleScript and other ways of accessing/talking to other applications). However, this happens transparently to the user. If the user drags a file on your application or its icon, or selects a file in an open panel, a temporary exception for this file is made.

It becomes part of your application’s sandbox (at least for a while — a restart of your application will exclude it from the sandbox again). Similarly, while your application can not communicate with other applications, the user can run an AppleScript in script editor, and *that* will be able to access all the applications it likes.

This means that, if a fake screen saver wants to e-mail your address book to an e-mail harvester, it will not be able to.

But my application uses AppleScript to …

This is a problem for those developers who want their application to run a script (e.g. automatically to import data from elsewhere). While I agree that’s annoying, a lot of the things developers do with AppleScript are workarounds for other issues (like the folder path example here). Chaining together applications is an engineer thing to do, and often causes bad usability. Users prefer a complete solution.

I hope many developers will consider banding together with developers of related tools, and do something not unlike Coda (Panic’s Transmit engine, The Coding Monkeys’ SubEthaEdit text engine). That avoids all the inter-application communication, gives you better control over the user experience (including sensible error messages instead of cryptic messages from AppleScript), and will probably make you Apple’s favorite child.

I can understand how Apple might want to bolt down security and instead provide dedicated API. Of course, if you’re the only one who needs the clicked Finder window’s folder, they might just not spare the manpower and you’re screwed. But I can understand their priorities, even if I only partially share them. And the above usability arguments may work as an encouragement for Apple to continue down this path.

Applications as entitlements

However, I do think that Apple should be flexible. Sandboxing will be mandatory to get applications approved for the Mac App Store in a few months. While you will be able to get exemptions for some of these restrictions, they are called “temporary”, which means what Apple giveth, Apple taketh away again. If they giveth at all.

Therefore, it would be great to have applications as entitlements: I.e. someone who needs to AppleScript the Finder could just add a com.apple.Finder entitlement, and then the user would get notified of that on installation. That way, if I install a screen saver and I get asked if this may access the address book, I know something is wrong. If it doesn’t ask for the address book entitlement, the address book API just wouldn’t work. Security. (Of course, there needs to be more granularity for the address book in particular, I would give applications access to my “Me” card, but not the rest of my address book – bad example).

The advantage of this approach is that Apple’s reviewers would just have to look at the entitlements to find out whether someone is doing something freaky. And if you used an unusual entitlement, Apple could request clarification, and then either require the use of better API, or reject, or accept.

It may still leave us at Apple’s approval mercy, but it is at least flexible enough to allow for many utilities to be re-added to the app store.

What can we do?

File bugs. If you like one of my suggestions above, feel free to request such behaviour from Apple. It will probably be marked as a duplicate, but it will get counted. But make sure you file the bug not just as a general mechanism, but in what way it applies to your application. When I talked to some Sandbox engineers at WWDC, they seemed very interested in accommodating our needs. Whether they will be able to do probably depends on what their superiors decide, but we have the engineers on our side. Even if you just write a short bug report, it will help. Besides, you can’t complain if you haven’t at least put your opinion on record.