Language Design

There are 11 entries in this Category.

Why Cocoa programmers can’t have nice things

IMG_0315
Amy Worrall pointed me at a nice post on the technical feasibility of using exceptions for more than programming errors in Cocoa by Hari Karam Singh. Even though he is misled by some Mac documentation into thinking iOS didn’t have zero-cost exceptions and then disproves that documentation by disassembly, he draws lots of valid conclusions.

However, the problem is not one of technology. The problem is one of the sheer size of Apple’s Cocoa codebase, which would have to be updated to survive having an exception thrown through it. Apple would have to add @trys in every location where they call a SEL, after all, since they don’t know which of them may be user-provided and throw.

Since they’re not doing that, a user who decides to use exceptions anyway would have to add @trys to every method that might ever be called by the framework. That means you can’t catch exceptions thrown by that method when you call it, though, because it swallows them itself. So if you want to handle errors from that method, you either split it up into okButtonClickedThrows: and okButtonClicked:, duplicating every method and working in parallel with two error handling schemes, or you give up, like Apple, and just use one non-exception error handling scheme.

I love exceptions, but I don’t think my Cocoa code will be cleaner and error handling nicer if I put a try block at the top of every action and delegate method. NSError is less dangerous, because if an object returns nil and gives an error (and you don’t look at the returned error), the method call will simply collapse (call to NIL is a no-op) so nothing much will happen. Since I can’t put up an error dialog from drawing code or table delegate callbacks like numberOfSections, there’s not much difference there. The code is actually cleaner, because with NSError and nil returns I can just ignore errors, while with exceptions in an exception-unsafe Cocoa, I must catch here or I’ll risk throwing through Cocoa.

C++ also has an advantage when working with exceptions over Objective-C because it uses “Resource Acquisition Is Initialization” (or RAII for short). Locks, allocations, even changes of a global boolean can be implemented as stack objects using RAII to set themselves up when created and clean up behind themselves in their destructor. You don’t even have a ‘finally’ block in the language. OTOH, every method you write in an exception-handling ObjC would need an @finally block, even if it doesn’t care about the errors, just to clean up behind itself.

ARC, @autoreleasepool and @synchronized can help a little with clean-up of memory and locks these days, as they’ll get triggered on an exception anyway. But as Cocoa and Apple’s frameworks currently stand, using exceptions effectively doubles your work.

The same applies to existing code. Nobody wants to have to completely rewrite their apps for 10.9 just to adopt a new error handling scheme when their code already has working error handling with NSError. Apple understands that their developers want a certain degree of backward compatibility. That’s the reason why only iOS got the new runtime on 32-bit: There was no code that relied on the old runtime there, it was a new platform. But all existing Mac applications would have been broken if the system had suddenly no longer guaranteed instance variable layouts and method lookup-tables. However, since 64-bit required changes to pointer sizes and data structures anyway, nobody complained when Apple introduced the new runtime for 64 bit on the Mac. They had to re-test and update their applications anyway.

All that said, I would love a new Objective-C framework that uses exceptions and is exception-safe for new projects to be built against. It just doesn’t seem like something Apple can retrofit at a whim.

At best, they can slowly make each framework exception-safe, and then in every spot where there can be an error, instead of returning it, look at an “app supports exception handling”-flag and throw their errors if that is set. That way, existing applications will keep working, while new applications can be written exception-safe. And once the majority of applications have moved to exceptions, Apple can switch to using exceptions themselves (see above — you don’t want 2 versions, exception-safe and unsafe, of every method), and tell the stragglers to please make their code exception-safe.

Posted on 2013-05-04 at 13:27 3 Comments

This post is filed under Language Design, Macintosh, Objective-C, Programming.

A proposal: Categories for C++

Categories?

One of the most useful features I’ve used in object-oriented programming languages is the “category”. A category is a way of adding methods to a class dynamically. Take, for example Objective C’s “NSAttributedString” class. As defined in the Foundation framework, it is simply a string where you can attach key-value pairs to a range of it.

Only a category in the GUI framework AppKit actually defines the methods and constants that define what key to use for “bold”, and let you draw such a string to the screen. Yet in everyday use with AppKit, these two disparate parts feel like one homogeneous part. And, more importantly, anyone who wants to write a text-processing command line tool can use NSAttributedString without having to drag in all that unneeded drawing code.

How C++ objects work

But C++ does not have this feature. So let’s come up with an implementation of this concept that a compiler vendor could implement, and that stays true to the core of C++, mainly it’s compile-time determination of as much as possible. But of course we want to be able to add a category to a system class or a class in another module, and want to define virtual methods in a category, and override them in a category on a subclass. So, of course we want to end up at something that looks like this in C++:

category BarSupport : MyObject // extend class MyObject with some methods.
{
    void doBar( MyObject baz );
};

and can be called just like any other method on MyObject, e.g.

MyObject foo, dodo;
foo.doBar( dodo );

If we didn’t have to support virtual methods, things would be trivial. A non-virtual method call like

MyObject foo;
foo.doBar(baz);

compiles to something like

MyObject_doBar( foo, baz );

Where the this pointer is simply passed in as the first parameter before the first parameter you defined. So all we’d have to do is tell the parser about these new methods, compile the functions that implement them, and call them.

But virtual methods work differently. First, there is a virtual function table, something like this:

struct MyObjectVTable
{
    void (*doFoo)( struct MyObject* this ); // defined by user as doFoo(void).
};

And whenever you define a new class, what it actually does is add a hidden instance variable at the start:

struct MyObject
{
    struct MyObjectVTable *vtable;
    int                   firstInstanceVariable;
};

And it declares a global variable containing the vtable once, for all objects created with this class, and stashes it in each new object’s vtable instance variable:

struct MyObjectVTable gMyObjectSharedVTable = { MyObject_doFoo };

...

// Equivalent code to MyObject* foo = new MyObject :
struct MyObject* foo = malloc(sizeof(struct MyObject));
foo->vtable = &gMyObjectSharedVTable;
MyObject_Constructor( foo );

Here, MyObject_doFoo() is a function just like our 

MyObject_doBar()

above. But when a virtual method is called, it is done slightly differently:

foo.vtable->doFoo( &foo );

This means that a subclass that wants to override doFoo() can provide its own gMySubclassSharedVTable that is also a struct MyObjectVTable, but contains a pointer to MySubclass_doFoo instead, and thus overrides the behaviour of doFoo() for all MySubclass objects. And if it wants to define additional virtual methods in its subclass, it can simply declare a struct MySubclassVTable that starts with the same fields in the same order as struct MyObjectVTable, and contains the additional methods after that. That way, anyone who expects an object for the base class doesn’t even have an inkling that there is more stuff after the methods it knows.

Applying that to Categories

Our categories also want to add methods, but the problem is we can’t just add new fields to the struct. The system classes like std::string have already been compiled, and we can’t just change their code to add these ivars. But what we can do is declare a parallel class hierarchy for our categories. So, first we declare a struct for our category:

struct MyObject_BarSupportVTable
{
    void (*doBar)( struct MyObject* this, struct MyObject* baz );
};

Then we extend the vtable to list categories:

struct CategoryEntry
{
    MyObject_BarSupportVTable* catVTable; // Simplified, each category's vtable is really a different struct.
};

struct MyObjectVTable
{
    struct CategoryEntry *cattable;
    void (*doFoo)( struct MyObject* this ); // defined by user as doFoo(void).
};

When a class is first built, the cattable array is empty, but as soon as someone declares a category, it gets added to that list. We just add some initialization code at the start of main() that mallocs the list. But how do we look up the vtable for a category? We could store its name, but then we’d have to loop over this list on each call and compare category names until we find it. How can we make that more efficient?

Simple: When we add the category to the class, we remember the index into the array where we added this category in the list, cache it in a global variable int gMyObject_BarSupport_Index; and then we can call a virtual method in a category like:

foo.vtable->cattable[gMyObject_BarSupport_Index].catVTable->doBar( &foo, baz );

Again, a subclass-category that overrides this method can just provide its own doBar method in the struct MyObject_BarSupportVTable.

The cost of categories

Of course this means that, just like with C++ virtual methods, and a bit more so, you pay a price for calling a virtual method in a category. You also pay with a little bit of overhead at startup, when the categories are added to the base class vtable. And just like with C++ virtual methods, to override a category method in a subclass, you have to know that the category exists on the base class. And if your program uses threads and you load a dynamic library that contains a category on a system class, bad things could happen while the category lists change under your running code’s rear.

Another gotcha with this approach is that the category list has to be built from the base class to the subclasses, because once a subclass locks down an index in the table for its first category vtable, you can’t add another category to the base class (it would have the index of the subclass’s category). But even that can be fixed:

We can just give every subclass its own categories table. So it only deals with the base class table when it wants to call a method inherited from the base class. E.g.:

struct MySubclassVTable
{
    struct CategoryEntry *cattable;
    void (*doFoo)( struct MyObject* this ); // defined by user as doFoo(void).
    struct CategoryEntry2 *cattable2;
    void (*doBar)( struct MyObject* this, struct MyObject baz ); // defined by user as doBar( MyObject baz ).
};

And now, calling a method in a category specific to the subclass simply goes through the cattable2.

Thoughts, suggestions, additional runtime geekery?

Posted on 2012-03-10 at 09:37 6 Comments

This post is filed under C++, Language Design, Programming.

Creativity Finds a Way

Uli's xDraw XCMD screenshot

Great observations

There is currently a nice little discussion on HyperCard going on in the comments on Stanislav Datskovskiy’s article Why HyperCard had to Die:

The article looks at the right facts, but I think draws the wrong conclusions: Yes, HyperCard was an echo of an era where a computer was a complex machine, and its owners were tinkerers who needed to customize it before it became useful. Yes, when Steve Jobs came back, he killed a lot of projects. And the Steve Jobs biography mentions that he doesn’t like other people screwing around with his designs.

But I do not think this automatically leads to the conclusion that Apple is on a grand crusade to remove the users’ control over their computers. Nor does it mean what many of the commenters say, that Apple is trying to dumb down programs and that programmers are underestimating their users.

How people work

Every programmer knows how important a coherent whole is: If a button appears in the wrong context, it will easily (and unintentionally) trick the user into thinking it does the opposite of what it really does. You can add paragraphs over paragraphs of text telling your users the opposite and they will not read it.

This is not because users are stupid, but because users “scan”. Screens are complex, and full of data. For the user to find something without spending hours of their life on it, they tend to quickly slide their eyes across the page, looking for words that come from the same category as the thing they are trying to do next.

This is a human efficiency optimization. It is a good thing. If we didn’t have this mechanism, we’d probably all be autistic, and incapable of coping with the world. Once a word is found, the user starts reading a little bit around it to verify the impression that this is what they want, and then they click the button.

It seems trivial to engineer a program for that, but it’s easy to overlook that a computer is not a single application at a time. There are other things happening on the screen, there may be other windows open. There may be system alerts popping up. Even if they are marked with each application’s icon or name, chances are that most users are too busy getting actual work done to memorize application names and icons. They won’t be able to distinguish what is your application, what is another.

Similar with haxies. Any halfway successful programmer probably has a story of how they tried to track down a crash or oddity a user encountered in their program that was actually caused by a plug-in or haxie that injects itself into every application to modify some behaviour system-wide. And once they are installed, even I occasionally forgot I had them installed. Or didn’t expect it to have an effect; Why should a tool that watches when my cursor hits the edge of my screen and then remote-controls the cursor on another computer as if it was an attached screen cause the menu bar to just randomly not show up when switching between applications?

Software is complex. Designing reliable, usable software is complex. In a comment, Stanislav had a great analogy for this (in response to someone’s pipe dream that one would just have to use HTML, and the technical stuff was all already done, you just had to add the human touch):

All the pieces of the world’s greatest statue are sitting inside a granite mountain. Somebody just has to come and chip away all the extra granite, adding the human touch. The technical problems are all virtually solved!

Software is hard. I don’t say this because it makes me sound cooler when I say I’m a programmer, but because you’re not just building a thing. You are building behaviours. HyperCard was notorious for being the tool for the creation of a number of the ugliest, least Mac-like programs ever released on the Mac. Because even with the best camera, your movie is only as good as the camera man.

So was Steve Jobs happy to get rid of HyperCard and stop people from screwing with his design? Probably. Was he forced to let it linger instead of killing it outright because he didn’t want to lose the educational market? I can’t disprove it. But Steve Jobs was also known to be an idealist. He genuinely thought his work would improve the world. What would he gain by making everyone dumb and uncreative?

Why assume malice when Occam’s Razor is a much better explanation?

You can’t hold a good idea down

When the Mac was originally released, it was intended as a machine for everyone. To bring computers to the masses. Almost from day one, the goal of Apple Computer, Inc. has been to drop the darned “Computer” from their name. Compared to the mainframes of the time, the Apple ][ that started the home computing revolution already was a “dumbing down” of computers.

Was this the end of the world? Should we have stayed in the trees? Will people become un-creative? Look around on the net. There are people out there who have no programming skills, who dig around in the games they bought and modify them, create their own levels, use existing game engines to create a game about their favorite book or TV show. Heck, there are people out there who create a 3D game engine in Excel.

If there is one thing we can learn, it is that Creativity Finds a Way.

HyperCard was designed in the late 1980s, for hardware of the time, for what very smart people thought would be the future at the time. Being creative with a computer, at the time, meant writing code. So they gave us a better programming language. Ways to click on a “Link to…” button to create the code to change pages. Not unlike Henry Ford’s competitors would have built you a better horse, but not a car.

Yes, I am saying that the developers of HyperCard didn’t quite anticipate the future correctly. They didn’t anticipate the internet, for example. That’s not a shame. It was ’87 back then. I didn’t get what the internet would be good for in ’91. I probably wouldn’t even have managed to invent a better horse. But anyway, all I am saying is that HyperCard’s creators didn’t know some things we know now, and probably made some compromises that wouldn’t make sense now.

The world has changed: This is 2011! All our programs do so much more. You can create 3D graphs in Excel, colorful drawings and animations in Keynote, and upload it all to the web with Sandvox. So many tools are available for such low prices. Why would you bother with a low-level, rudimentary tool like HyperCard when all you want to do is a movie with some branching?

A new tool for a new world

After all that, it might surprise you that I still agree with everyone in the comments who says that we need a new HyperCard for the 2010s. However, I do not agree that any of the examples the commenters mentioned (or even HyperCard as it shipped) are this program. Yes, Xcode and the NeXT-descended dev tools, and VB and others use the Rapid Application Development drag-and-drop manipulation to lay out your UI. But guess what? So does Pages.

Yes, you can use Ruby and Python and Smalltalk to branch between different choices. Or you could just use links to move between web pages built using Sandvox.

Yes, you can build real, runnable applications from your work with Java or AppleScript. But why would anyone want to build an application? Movies can be uploaded to YouTube, web sites can be built with WordPress, and I don’t have to transfer huge files to users. I just send my friends the link, and they know what to do. There’s no installer.

Our computing world has become so much richer, so much easier, that it is more efficient and actually smarter to just create your stuff with those tools that we old HyperCarders see as dumb. They can stand on the shoulders of giants, and spend their time creating the best possible gameplay instead of coding yet another 3D renderer. That is why HyperCard 2.4 just won’t cut it, or as David Stevens commented on that very same article:

most people get on a train to go somewhere, not because they really want to lay track, which explains the shortage of track laying machines in yard sales, and the demise of HyperCard.

The new HyperCard won’t be like HyperCard. Maybe the web is enough. Maybe it will just be a good “web editor”, like it used to be included in every copy of Netscape back in the day.

Or maybe, it will just be a niche product aimed at people who find that they want to do more than their tools let them do. This will not be the typical movie-maker, podcaster or writer. Like the directors, radio hosts or journalists in the generations before them, those will specialize. They will be exceptional at directing, making a show or researching the truth. But they will not care how the camera transports the film, they won’t care how their voice is really broadcast as radio waves and re-assembled in the receiver, nor how to build a printing press.

The people a new HyperCard is aimed at will be a person like you, who saw HyperCard, and at some point stood up and said: This is not enough. I want to create more. And then maybe went out and bought CompileIt!, which let her use the system APIs from the comfort of her HyperCard stack, only needing to use the scary memory management stuff when absolutely necessary. And then went and bought MPW, or THINK C, or CodeWarrior, or Xcode.

A real programmer doesn’t program because she wants to use HyperCard. A real programmer programs because she wants to. Because she just has to. A real programmer doesn’t limit herself to that one language and IDE she learned once so she never has to learn anything else. A real programmer learns new programming languages because each language teaches her a new way of looking at or solving a problem. A real programmer has been programming since she opened PowerPoint for the first time. She will find a way.

It’s been like that back in the days of HyperCard. Why shouldn’t it be like that again?

Posted on 2011-12-02 at 09:45 Leave a Comment

This post is filed under Language Design, Macintosh, Programming, Usability.

Dennis Ritchie Deceased

Apparently, a few days ago, Dennis Ritchie, the “R” in “K&R”, co-creator of the C programming language has died.

Thank you for laying the groundwork for our profession, Mr. Ritchie.

Posted on 2011-10-13 at 07:40 Leave a Comment

This post is filed under Language Design, Programming.

Funny thing about C parameter evaluation order…

I just explained this to a friend today, and thought this might make an interesting blog posting:

#include <stdio.h>
int main( int argc, const char * argv[] )
{
	char	theText[2] = { 'A', 'B' };
	char*	myString = theText;
	printf( "%c, %c\n", *(++myString), *myString );
	return 0;
}

The above code is platform-dependent in C. Yes, you read correctly: platform dependent. And I’m not nitpicking that this may cause a problem if your compiler is old or that some compiler may not have printf() or the POSIX standard.

This code is platform-dependent, because the C standard says that there is no guarantee in which order the parameters of a function call get evaluated. So, if you run the above code, it could print B, B (which most of you probably expected because it corresponds to our left-to-right reading order) or it could print B, A.

If you want to test this and you own an Intel Mac, you can do the following thanks to Rosetta’s PowerPC emulation: Create a new “Standard Tool” project in Xcode and paste the above code into the main.c file. Switch to “Release” and change “Architectures” in the build settings for the release build configuration to be “ppc”. Build and Run. It’ll print B, B. Now change the architecture to “i386″ and build and run again. It’ll print B, A.

So, why doesn’t C define an order? Why did anyone think such odd behaviour was a good idea? Well, to explain that, we’ll have to look at what your computer does under the hood to execute a function call. In general, there are two steps: First, the parameters are evaluated and stored in some standardized place where the called function can find them, and then the processor “jumps” to the first command in the new function and starts executing it.

Some CPUs have registers inside the CPU, which are little variables that can hold short values, and which can be accessed a lot quicker than actually going over to a RAM chip and fetching a value. There are different registers for different kinds of values. Many CPUs have separate registers for floating-point numbers and integers. And just like with RAM, it’s sometimes faster to access these registers in a certain order.

So, it may be faster to first evaluate all integer-value parameters, and then those that contain floating-point values. Depending on what physical CPU your computer has (or in the case of Rosetta, what characteristics the emulated CPU your code is being run on has), these performance characteristics may be different. Some CPUs may have so few registers that the parameters will always have to be passed in RAM. Others may put larger parameters in RAM and smaller ones in registers, others again may put the first couple parameters in registers (maybe even distributing a longer parameter across several registers), and the rest that don’t fit in RAM, etc.

So, to make sure C can be made to run that little bit faster on any of these CPUs, its designers decided not to enforce an order for execution of parameters. And that’s one of the dangers of writing code in C++ or Objective C: It may look like a high-level language, but underneath it is still a portable assembler, with platform-dependencies like this.

Posted on 2007-10-19 at 11:37 3 Comments

This post is filed under Language Design, Macintosh, Programming and tagged , , , , , , , , , .

Generating Machine Code at Runtime

Okay, so my next attempt at learning how my computer works and how to speak machine language is the following C code fragment:

typedef int (*FuncPtr)();

// Create a function:
char            testFunc[] = { 0x90,                         // NOP (not really necessary...)
                               0xB8, 0x10, 0x00, 0x00, 0x00, // MOVL $16,%eax
                               0xC3 };                       // RET

// Make a copy on the heap, OS doesn't like executing the stack:
FuncPtr         testFuncPtr = (FuncPtr) malloc(7);
memmove( (void*) testFuncPtr, testFunc, 7 );

printf("Before function.\n");
int result = (*testFuncPtr)();
printf("Result %d\n", result);

Basically, this stores the raw opcodes of a function in an array of chars. The first byte of each line is usually the opcode, i.e. 0x90 is No-Op, 0xB8 is a MOVL into the eax register (with the next 4 bytes being the number to store, in this case 16), and 0xC3 is the return instruction (I had to look up the opcodes in Intel’s documentation).

One thing to watch out for here (at least on Mac OS X), is that you’ll get a bad access error if you try to execute testFunc directly. That’s because testFunc is on the stack, and the stack shouldn’t contain executable code (it’s a small safety measure). So, what we do is we simply malloc some memory on the heap, and stuff our code in there.

You may wonder why I’m using eax of all registers to store my number 16 in. Easy: Because the convention is that an int return value (and most other 4-byte return values) goes in eax when a function returns. So, what this does is it essentially returns 16. Which our printf() proves. Neat!

Intel’s documentation describes the opcodes in a very complicated way, so what I essentially do is I write some assembler code and enclose the instruction whose byte sequence I want to find out in instructions whose byte sequence I already know (I like to use six nops, which are short and show up as 0x90 90 90 90 90 90). Then I compile that, and then use a hex editor to search for the known instructions, and whatever is between them must be my new one. Here’s a small table of other operations you may find in the typical program and what byte sequences they turn to:

0×50 pushl %eax
0×53 pushl %ebx
0×55 pushl %ebp
0×89 E5 movl %esp, %ebp
0×90 nop
0xB8 NN NN NN NN movl $N, %eax
0×68 NN NN NN NN pushl $N
0xE8 NN NN NN NN call relativeOffsetNFromEndOfInstruction
0x8B 1C 24 movl (%esp), %ebx
0x8D 83 NN NN NN NN leal relativeOffsetToData(%ebx), %eax
0x8D 85 NN NN NN NN leal relativeOffsetToData(%ebp), %eax
0x5B popl %ebx
0×83 C4 NN addl $NN,%esp
0×83 EC NN subl $NN,%esp
0x8B 00 movl (%eax), %eax
0×89 45 NN movl %eax, NN(%ebp)
0xC9 leave
0xC3 ret

The code fragment above is essentially what one would need to create a just-in-time compiler. For a real compiler, instead of executing this directly, we’d have to write it to a complete MachO file and link it with crt1.o.

Update: on top of the instructions for position-independent code (PIC), I’ve also added some more useful in passing structs as parameters on the stack.

Posted on 2007-05-01 at 12:53 Leave a Comment

This post is filed under Assembler, Language Design, Macintosh, Programming and tagged , , , , , , , , , , .

Building a loader…

One neat thing about a computer’s internals is the “loader”. A loader is a little bit of code (generally in the system) that takes a file of compiled code and loads it into RAM, preparing it for execution.

Preparing code for execution means that it looks where the system functions are and fixes up any calls to them to point to the current address, and does the same for other dynamically loaded libraries that might not get loaded at the same address every time.

For my experiments with code generation, runtimes etc., I recently wrote myself a little loader. I could have used the system’s loader, but then my output files would have had to adhere to the Mach-O file format, and that looked a tad too complicated for me. So, I rolled my own that’s fairly simple, but in essence does the same things a real loader would do:

It loads the actual code and data from a file. Then it looks at the symbol table which is also part of the loaded data, and looks up the actual function corresponding to each symbol name, and inserts it in the symbol table. The way one generally does this is to have the symbol table consist of 5-byte placeholders. Why 5 bytes? Well, because that’s the size of a JMP (jump-to-address) instruction in Intel machine code.

[Illustration of the code, the symbol table entry and how control flows when it is called]

All code that calls the function is written so it jumps to these 5 bytes and executes them, using a CALL instruction. And when the loader prepares our code for execution, it fills these 5 bytes with the opcode for the JMP instruction plus the address of the actual function to be called (in the case of our example image, that would be the address of printf()).

The nice part about this approach is that all CALL statements point to this central JMP statement in the symbol table (which is often called a “jump island”), and the loader only has to update this one place. Another nice part comes from the use of CALL and JMP: CALL remembers the location we were at when it got executed, so that the called function can return to that place. On the other hand, the JMP instruction just transfers execution to its destination. That means that when printf returns, it will not return to the JMP instruction, but rather it will directly jump back to the CALL instruction, because that was the last time someone saved where to return to. So, we don’t need an additional RET (return) instruction after the JMP, and we don’t waste time jumping to the symbol table just to jump back to the CALL statement.

But still, we make that extra jump. Can’t we do better? Yes! Since CALL pushes the address of the instruction after it onto the stack, it’s easy to find the CALL instruction that called into the jump island. So, many loaders don’t put the address of printf into that symbol table entry, but rather the address of a linker-fix-up-function. This function then inserts the address of printf into the CALL instruction. That way, the first time each CALL statement is executed, we go through the jump island, the fix-up function changes the CALL instruction to call printf directly and then jumps to printf. Subsequent times, the CALL statement will call printf directly.

So, how does the fix-up function know to call printf? Well, before it updates the CALL statement, it takes the address of our jump island from there. So, we can just use the return address to find the CALL statement, and the CALL statement to find our jump island, and then do whatever we would have done before to get the real function address (e.g. get the function’s name that our compiler stored before our jump island in the code), stash it in the CALL statement and jump to it ourselves.

The advantage is that we can lazily update each CALL to directly call printf, which saves us one extra jump (and maybe even flushing memory — who knows how far the symbol table is away from the code we’re executing). But since we do it only on code that is actually called, if a piece of code is never reached, we never look up that symbol. That may save us a lot of time. It’s also handy for weak-linking: If the function is never called, we don’t care whether the library isn’t available. So, whoever wrote the code we’re loading can just check for existence of a library and not call into it if it isn’t there.

Neat, huh?

Posted on 2007-04-28 at 11:33 Leave a Comment

This post is filed under Language Design, Programming and tagged , , , .

How xTalk Syntax should be

When I talked to Doug Simons at the WWDC Apple Campus Bash, one thing he said that I hadn’t even realised myself yet was that he had the impression I had a very clear picture of how a modern HyperCard and HyperTalk had to be. Thinking about it a little, I realised that he was right, and I also realised that I hadn’t written about it yet. So, I’m going to do that now, as far as I have managed to put it in words so far:

One of my main sources of inspiration on how HyperTalk should look, incidentally, comes from AppleScript. AppleScript allows for multi-word commands and methods, so it really should be every xTalker’s wet dream. Instead, it constantly complains about data types, and it lets you write sentences that simply aren’t English (and I’m not talking about the French AppleScript dialect here…), and because it’s a modular language into which every application plugs its own set of commands and syntax, it can be horribly inconsistent. Which is something none of the AppleScript developers can be blamed for.

So, here’s a few rules I’ve come up with for creating “good” HyperTalk syntax:

  1. Make your commands, functions and identifiers correct grammatical English, or at least some sort of telegram-style English. This mainly boils down to making commands imperative sentences and making your parameter labels read like parts of a sentence and not just like lists of … stuff. The most egregious violator of this rule in HyperTalk itself is probably the answer command, or more precisely, its variants answer file and answer program. That’s also where the clones picked up some ugliness for commands like alert.
    Why? Because people need to memorise the syntax, and we already have cryptic languages that have arbitrarily restricted syntax. The point of xTalks should be to try to get rid of as many restrictions as we can, so users just have to remember the vague sentence content and the language skills they’ve been honing all through elementary school will take care of the rest.
  2. Another rule that I’m still trying to work out the full repercussions of is that HyperTalk has no data types. Some people phrased this as everything is a string, which is probably a rule that works most of the time. The thing is, in the real world things actually do have types, it’s just that they don’t have just one type, they have several at once. (I’m talking about the image presented to the user – it’s fine if you store values as typed data internally to make things go faster.)
    Why? Think of text: When you write text on a sheet of paper, not only is it made up of a string of characters, it may also contain numbers, tables and even graphics. Furthermore, it’s a physical object, a piece of paper, which can have a colour, texture, shape… lots of additional cues that may or may not be relevant to your use right now. However, – and that’s the sticking point – our use of real-world objects changes over time. One moment a newspaper can be a source of information, the next it is a makeshift fly-swatter, the next it’s used for wrapping gifts.
    So really, that some clones show associative arrays as empty when used as a string may be wrong. Maybe we should really show a string representation instead, maybe XML or old-style Plist format or whatever. Maybe all the user should see would be good old delimited item lists with specialised syntax that takes care for you to escape commas and stuff?
    I don’t have absolute answers, but the point is that real life really deals in multiple selective views on the same data, which is why so many programmers fall in love with the Model-View-Controller paradigm. Maybe we should now take this abstract model that seems to represent the world so well and bring it back into the real world and offer an easily understood analogy?
  3. Give everything a strict, logical syntax. The problem with Point #1 above is that a computer really doesn’t understand natural language. So, everything we’re doing to make stuff more realistic beyond a certain point ends up making the experience more frustrating. In robotics this is called the uncanny valley, and Drunkenbatman already applied it to AppleScript.
    Basically, if you make a script sound too English, people will not be able to discern the rules it works by anymore, and will just throw real, complex English at it, and constantly be disappointed by the wrong answers and error messages they get.
    Take, for example, Inform 7. It has a beautiful, English language that lets you describe rooms and complex actors and objects in a text adventure, but sometimes it will just tell you it can’t understand a perfectly valid English sentence and leave you to figure out what you did wrong. So, consistency helps, but where it doesn’t, you’d better have a syntax table somewhere so people can look up the rules if they feel they need to.
    Due to this problem, it can be really helpful to try and stick to a clearly-defined subset of real English that is coherent and lets your users discover additional features of one command just through their knowledge of another. Most of HyperCard’s commands have the form
    <verb> <value> <preposition> <value>
    where preposition and value are repeated several times. An xTalk shouldn’t try to be too literary.
  4. Things the user can work with should be objects. HyperCard 2.0 was probably the first to break this rule: Instead of having menu and menu item objects in the stack, like SuperCard did it, menus were added as weird transient objects that the scripter has to create in code. Everything should be a real object. Not just movies and movie players on a card, even parts of other objects: The menu and its items in a popup button, images embedded in text, even characters of text in a field should be accessible via chunk expressions and changeable through property syntax.

Posted on 2006-08-23 at 13:13 Leave a Comment

This post is filed under Language Design and tagged , , , , , , , .

Inform 7 (IF Language) is out!

Mr. Stoneship just posted his del.icio.us link to Inform 7, the interactive fiction development language. I don’t know how such a cool program could so quietly creep up on us. It’s a very interesting approach at a GUI with an English-like programming language and lots of new ideas:

There are several views on your game, with a transcript view that can perform something akin to unit tests on your game output, the game is written pretty much as english text, you have a tree view and transcript of actions that can be re-run, and you can switch between a running game and a game being designed and do something like “fix and continue”.

The only trouble is, it doesn’t just have the same name, but also the same file extension as the Inform drawing program… I sent out messages to the authors, hopefully they’ll work something out.

Posted on 2006-05-06 at 13:20 Leave a Comment

This post is filed under Language Design and tagged , , , , .

Headaches further Revelations

Sometimes you sit there with a headache and the memory of an article and two e-mails you read two days ago, not being able to do much else, and you suddenly comprehend something you didn’t before. And if the headache isn’t from alcohol, you can actually assume that this revelation will still be valid once you feel human again. Try this one:

There were a couple of postings on Cocoa-Dev by someone trying to use Cocoa APIs using AppleScript Studio’s call method function. Now, if you remember, that function lets people used to not-quite-English statements like

copy (display dialog with prompt "I failed jiggery-pokery but I had A's in hullaballoo") to myVar

turn nice readable Objective C statements like

foo = [[NSFileManager defaultManager] directoryContentsAtPath: @"/Users/"];

into toll-free-bridged unreadabilities like

set foo to (call method "directoryContentsAtPath:" of (call method "defaultManager" of class
"NSFileManager") with parameter "Macintosh HD:Users")

(Note that I inserted the brackets in the AppleScript to improve readability).

Now, when I first read this I thought that guy got what he deserved. If you clicked the link above, you’ll know my thoughts about AppleScript, and this example shows beautifully why you want to do Cocoa development in Objective C. So why doesn’t that guy bite the bullet and switch languages?

Then, today, I wrote an e-mail to Tom Pittman and in the last line gushed a little about how his CompileIt! stack helped me get into programming from within the safety of HyperCard. Now, for those of you who never got to play with HyperCard or CompileIt!, basically CompileIt is like AppleScript Studio’s call method command: It lets you call all the system APIs as if they were HyperTalk handlers and functions. So to have Quickdraw draw a rectangle, you’d write something like:

put newPtr(8) into myRect -- 8 bytes, for 4 two-byte shorts
setRect myRect,10,10,100,100
frameRect myRect
disposPtr myRect -- yes, no "e" - those were the System 6 symbol names.

Of course, you could have just done this in plain HyperTalk, by scripting the user’s actions with the drawing tools, like:

choose rect tool
drag from 10,10 to 100,100

but play along there, will you? Well, anyway, just like with ASS and call method, this allows you to turn a straight affair of English-like code into a bastardised form of some systems-programming language.

So, what’s the benefit? The benefit is a shallower learning curve. With tools like HyperCard or ASS, you can create the basics of an application, all the drudgework, using a much more natural set of metaphors. And when those fail you, you don’t have to start from scratch writing event loops and window-management code. You just learn about those few lower-level Toolbox or Cocoa commands you need and only use those.

The advantage is that you can get to know the frameworks step by step, having small successes each time. By the time you have to learn the actual details of Cocoa programming, you’ll already know most of the commands and conventions, and it won’t feel half as foreign as it would have if you’d jumped straight in.

It doesn’t change my opinion that AppleScript is a bass-ackwards language, and it doesn’t change my opinion that call method‘s main effect is making code unreadable, but at least it makes me frightened of all these AppleScripters that will push into the Mac programming market eventually once they’ve moved to a better-designed language. They had the same basic learning curve I had, and they put up with AS… They’ll smoke me in their pipes… :-)

Posted on 2006-04-07 at 13:01 Leave a Comment

This post is filed under Language Design, Macintosh, Objective-C, Programming and tagged , , , , .

More »