subscribe via RSS

  • Using Jekyll for static web site building on a Mac

    Martin Pilkington occasionally mentioned the Jekyll static web site builder in his tweets, so when I wanted to start a new web site, I thought I'd give it a try, as I've been uncomfortable with having to back up WordPress sites in separate steps to get both the database and the image and movie assets.

    Installing Jekyll

    Make sure you have Xcode installed and its command line tools. Then just open Terminal and type:

    sudo gem install jekyll

    Creating a new site

    To create a site, do

    jenkins new

    This creates a folder named in the current directory to hold the files for the new site.

    Open the folder and in it the _config.yml file. Change the entry title: to what you want to call your new site, and url: to your domain name (in our above example, Feel free to change what few other settings there are, but you don't need to.

    Two neat _config.yml tips:

    1. You can leave entries empty, e.g. the email: or github_username:, and they will just disappear, including their icons.
    2. You can make files and folders that Jekyll would usually skip (like .htaccess) by adding an entry like: include: [.htaccess, .well-known, _foo.xml]. You can also exclude files from copying/processing this way, e.g. exclude:

    Testing the site

    Jekyll includes its own web server. Simply type

    jekyll serve

    the site is now available under http://localhost:4000. The server will automatically watch for changes to the folder and re-build the site. If you still need to manually trigger a rebuild (e.g. to deploy your site without launching the server), just use

    jekyll build

    Adding your own pages

    Like WordPress, Jekyll has pages and blog posts. Any file that doesn't start with an underscore is considered as a page. Be it index.html or Let's edit to describe our site, not Jekyll. Open it in a text editor.

    The file starts with a section like this:

    layout: page
    title: About
    permalink: /about/

    called the "front matter", followed by regular Markdown, as you'd expect from a .md file. This section specifies the info that Jekyll needs about the page, and tells Jekyll to substitute placeholders in the file. You can leave this section empty, just its presence tells Jekyll to process the file. So it's easy to create a new page. You can also make up your own settings here, if you want.

    The ones in this example are standard ones Jekyll knows out of the box:

    layout: specifies that the file _layouts/page.html should be wrapped around this file, and this file's contents should be inserted where that file says . This is how Jekyll applies themes and shows navigation on each page.

    permalink specifies the address at which the page will end up in the generated web site. So in our example, you'd find this page at instead of at

    title: is actually just a variable used by the _layouts/default.html template. Any variable you define can be used on the page by writing e.g. Using Jekyll for static web site building on a Mac. So if you added a line temperature: 40 Centigrade you could put it on the page as .

    Other interesting variables are categories, tags and published.

    Blogging with Jekyll

    Jekyll offers special blogging support. Mainly this just involves saving pages into the _posts folder and prefixing the file names with an ISO date, e.g. 2015-01-30-My First But it also has special functions to make it easier to link between posts in a stable fashion, and to generate lists of posts with teaser text etc. The Official Jekyll docs on blogging cover this well.

    Importing from WordPress

    Jekyll has support for importing from WordPress. First, install the importer:

    sudo gem install jekyll-import
    sudo gem install sequel
    sudo gem install unidecode
    sudo gem install htmlentities
    sudo gem install mysql2

    and then do what Jekyll's WordPress Importer docsy say.

    If you're on DreamHost, note that you need to go to "MySQL Databases" in the Panel, and there under "Database(s) on this server:" click the user name under "Users with Access" and enter your current IP address under "Allowable Hosts" before you can run this as that user. In this section you can also look up the (randomly generated) password DreamHost's One-click install may have given your database.

    Then go to "Hostnames for this MySQL server:" again and click "phpMyAdmin" behind your server's name and look up what prefix DreamHost used for your database. The example in the Jekyll importer's docs uses "wp_" but DreamHost adds a random element like "07foo_" (e.g. "wp_07foo_posts" instead of "wp_posts").

  • Handling keypresses in Cocoa games

    At first blush, Keyboard event handling for games in Cocoa seems easy: You add -acceptsFirstResponder and -becomeFirstResponder overrides to your custom game map view, then override -moveUp:, -moveDown:, -moveLeft: and -moveRight: to handle the arrow keys.

    However, if you play a game like that, you'll notice one big difference to most other games: It only ever accepts one keypress at a time. So if you're holding down the up arrow key to have your character run forward, then quickly press the right arrow key to sidestep and obstacle, your character will stop in its tracks, as if you had released the up arrow key.

    This makes sense for text entry, where you might accidentally still be holding down one character while another finger presses the next, but for a game this is annoying. You want to be able to chord arbitrary key combinations together.

    I found a clever solution for game keyboard handling on the CocoaDev Wiki, but it's a bit old and incomplete, so I thought I'd provide an updated technique:

    The solution is to keep track of which key is down yourself. Override -keyDown and -keyUp to keep track of which keys are being held down. I'm using a C++ unordered_set for that, but an Objective-C NSIndexSet would work just as well:

    @interface ICGMapView : NSView
    	std::unordered_set	pressedKeys;

    and in the implementation:

    -(void)	keyDown:(NSEvent *)theEvent
    	NSString	*	pressedKeyString = theEvent.charactersIgnoringModifiers;
    	unichar			pressedKey = (pressedKeyString.length > 0) ? [pressedKeyString characterAtIndex: 0] : 0;
    	if( pressedKey )
    		pressedKeys.insert( pressedKey );
    -(void)	keyUp:(NSEvent *)theEvent
    	NSString	*	pressedKeyString = theEvent.charactersIgnoringModifiers;
    	unichar			pressedKey = (pressedKeyString.length > 0) ? [pressedKeyString characterAtIndex: 0] : 0;
    	if( pressedKey )
    		auto foundKey = pressedKeys.find( pressedKey );
    		if( foundKey != pressedKeys.end() )

    Of course, you'll also want to react to modifier keys, and like most games, you will want to treat them not as modifiers in a shortcut, but as regular keys, so people can press Command to fire, or so. That's basically the same, just that you override -flagsChanged: and that there are no standard character constants for the modifier keys. So let's just define our own:

    // We need key codes under which to save the modifiers in our "keys pressed"
    //	table. We must pick characters that are unlikely to be on any real keyboard.
    //	So we pick the Unicode glyphs that correspond to the symbols on these keys.
    	ICGShiftFunctionKey			= 0x21E7,	// -> NSShiftKeyMask
    	ICGAlphaShiftFunctionKey	= 0x21EA,	// -> NSAlphaShiftKeyMask
    	ICGAlternateFunctionKey		= 0x2325,	// -> NSAlternateKeyMask
    	ICGControlFunctionKey		= 0x2303,	// -> NSControlKeyMask
    	ICGCommandFunctionKey		= 0x2318	// -> NSCommandKeyMask
    -(void)	flagsChanged: (NSEvent *)theEvent
    	if( theEvent.modifierFlags & NSShiftKeyMask )
    		pressedKeys.insert( ICGShiftFunctionKey );
    		auto foundKey = pressedKeys.find( ICGShiftFunctionKey );
    		if( foundKey != pressedKeys.end() )
    	if( theEvent.modifierFlags & NSAlphaShiftKeyMask )
    		pressedKeys.insert( ICGAlphaShiftFunctionKey );
    		auto foundKey = pressedKeys.find( ICGAlphaShiftFunctionKey );
    		if( foundKey != pressedKeys.end() )
    	if( theEvent.modifierFlags & NSControlKeyMask )
    		pressedKeys.insert( ICGControlFunctionKey );
    		auto foundKey = pressedKeys.find( ICGControlFunctionKey );
    		if( foundKey != pressedKeys.end() )
    	if( theEvent.modifierFlags & NSCommandKeyMask )
    		pressedKeys.insert( ICGCommandFunctionKey );
    		auto foundKey = pressedKeys.find( ICGCommandFunctionKey );
    		if( foundKey != pressedKeys.end() )
    	if( theEvent.modifierFlags & NSAlternateKeyMask )
    		pressedKeys.insert( ICGAlternateFunctionKey );
    		auto foundKey = pressedKeys.find( ICGAlternateFunctionKey );
    		if( foundKey != pressedKeys.end() )

    An alternative would be to just enlarge the numeric type used to store keys in your unordered_set. Instead of two-byte unichar values, you'd just pick uint32_t, and then define the constants as values that are out of range for an actual unichar, like 0xffff1234. If you're using NSIndexSet, you're lucky, it uses NSInteger, which is already larger.

    Then add an NSTimer to your class that periodically checks whether there are any keys pressed, and if they are, reacts to them:

    -(void) dispatchPressedKeys: (NSTimer*)sender
    	BOOL	shiftKeyDown = pressedKeys.find(ICGShiftFunctionKey) != pressedKeys.end();
    	for( unichar pressedKey : pressedKeys )
    		switch( pressedKey )
    			case 'w':
    				[self moveUp: self fast: shiftKeyDown];

    Since your timer is polling at an interval here, and you can't make that interval too fast because it's the rate at which key repeats will be sent, it is theoretically possible that you would lose keypresses whose duration is shorter than your timer interval. To avoid that, you could store a struct in an array instead of just the keypress in a set. This struct would remember when the key was originally pressed down, and when the last key event was sent out.

    That way, when the user begins holding down a key, you'd immediately trigger processing of this key once, and make note of when that happened. From then on, your -dispatchPressedKeys: method would check whether it's been long enough since the last time it processed that particular key, and would send key repeats for each key that is due. As a bonus, when a key is released, you could also notify yourself of that.

    You could even create "key event" objects of some sort to hand into your engine.

  • Adding Lua 5.2 to your application

    Note: The code in here is adapted from an actual project, however I've not yet had time to verify it doesn't have typos. Give me a weekend or two. I just found so little information on the new version I wanted to dump this to the web now in the hopes of being at least vaguely helpful.

    Lua is a really cool, clean little programming language that is easy to embed in your applications. Not only is it under a permissive license, it's ANSI C.

    However, recent updates have made most of the documentation about it on the web a bit outdated, so I thought I'd drop this quick tutorial on how to add Lua to your application and do some of the typical inter-operation things with it that you'd want to do when hosting scripts in your application.

    Building Lua

    Building Lua is very easy. After getting the source code, you duplicate the file lua/src/luaconf.h.orig under the name lua/src/luaconf.h. Then you point Terminal at Lua's folder and do

    make macosx test

    (Or if you're not on a Mac, use the appropriate platform name here, you can see available ones by just calling make without parameters in that folder)

    This will churn a short moment, and then you'll have a liblua.a file. Add that to your Xcode project (or equivalent) so it gets linked in, and make sure the header search paths include the lua/src/ folder. That's it, now you can use Lua in your application.

    Running a Lua script

    To use Lua, you include the following headers:

    #include "lua.h"
    #include "lauxlib.h"
    #include "lualib.h"

    (If you're using C++, be sure to wrap them in extern "C" or you'll get link errors) Then you can simply compile the following code to initialize a Lua context and run a script from a text file:

    lua_State *L = luaL_newstate();	// Create a context.
    luaL_openlibs(L);	// Load Lua standard library.
    // Load the file:
    int s = luaL_loadfile( L, "/path/to/file.lua" );
    if( s == 0 )
    	// Run it, with 0 params, accepting an arbitrary number of return values.
    	//	Last 0 is error handler Lua function's stack index, or 0 to ignore.
    	s = lua_pcall(L, 0, LUA_MULTRET, 0);
    // Was an error? Get error message off the stack and send it back:
    if( s != 0 )
    	printf("Error: %s\n", lua_tostring(L, -1) );
    	lua_pop(L, 1); // Remove error message from stack.
    lua_close(L);	// Dispose of the script context.

    The script file would contain something like:

    -- this is a comment
    io.write("Hello world, from ",_VERSION,"!\n")

    Now you can run a file full of commands. But how do you have it call back into your application? There's a special call for that, lua_register, which creates a new function that actually wraps a special C function. You call it like this:

    // Create a C-backed Lua function, myavg():
    lua_register( L, "myavg", foo );	// Create a global named "myavg" and stash an unnamed function with C function "foo" as its implementation in it.

    to register a C function named foo as a Lua function named myavg. The actual function would look like this:

    // An example C function that we call from Lua:
    static int foo (lua_State *L)
    	int n = lua_gettop(L);    /* number of arguments */
    	lua_Number sum = 0;
    	int i;
    	for (i = 1; i <= n; i++)
    		if (!lua_isnumber(L, i))
    			lua_pushstring(L, "incorrect argument");
    		sum += lua_tonumber(L, i);
    	lua_pushnumber(L, sum/n);        /* first result */
    	lua_pushnumber(L, sum);         /* second result */
    	return 2;                   /* number of results */

    This example function loops over all parameters that have been passed (using lua_isnumber to check they're numbers, and lua_tonumber to actually retrieve them as ints), which may be a variable number, adds and averages them, and then pushes two return values on the stack (the average and the sum), and returns the number of return values it gave.

    You could now call it like:

    io.write( "Average is: ", myavg(1,2,3,4,5) )

    from Lua. The funny thing here is, in Lua, there are no functions in the traditional sense. It's a prototype-based programming language, so all functions are closures/blocks/lambdas, and can be treated just like any value, like an integer or a string. To declare a function, lua_register simply creates a global variable named myavg and sticks such a function object in it.

    When you declare a function in Lua, it's also really just a shorthand for an assignment statement. So to run a function declared in a Lua file, like:

    function main( magicNumber )
        io.write("Main was called with magicNumber ", magicNumber, "!")

    you first have to execute it, which will create the global named main and stick a function in it. Only now do you look up the function object from that global and call it, again using lua_pcall like here:

    s = lua_pcall(L, 1, LUA_MULTRET, 0);	// Tell Lua to expect 1 param & run it.

    The 2nd parameter to lua_pcall tells it how many parameters to expect. Objects are likewise just tables (i.e. key-value dictionaries) where ivars are just values, and methods are functions stored as values. So, to create a new object with methods implemented in C, you do:

    // Create a C-backed Lua object:
    lua_newtable( L );	// Create a new object & push it on the stack.
    // Define mymath.durchschnitt() for averaging numbers:
    lua_pushcfunction( L, foo );	// Create an (unnamed) function with C function "foo" as the implementation.
    lua_setfield( L, -2, "durchschnitt" );	// Pop the function off the back of the stack and into the object (-2 == penultimate object on stack) using the key "durchschnitt" (i.e. method name).
    lua_setglobal( L, "mymath" );	// Pop the object off the stack into a global named "mymath".

    To call this, function, you do it analogous to before, just that you first use lua_getglobal( L, "mymath" ) to push the object on the stack, then lua_getfield to actually push the "durchschnitt" function stored under that key in the object.

    Since functions are closures/blocks/lambdas, they can also capture variables ("upvalues"). To set those, you use lua_pushcclosure instead of lua_pushcfunction and pass the number of values you pushed on the stack to capture as the last parameter. E.g. if you wanted to pass along a pointer to an object in your program that the session object wraps, instead of stashing it in an ivar, you could capture it like:

    // Define session.write() for sending a reply back to the client:
    lua_pushlightuserdata( L, sessionPtr );	// Create a value wrapping a pointer to a C++ object (this would be dangerous if we let the script run longer than the object was around).
    lua_pushcclosure( L, session_write, 1 );// Create an (unnamed) function with C function "session_write" as the implementation and one associated value (think "captured variable", our userdata on the back of the stack).
    lua_setfield( L, -2, "write" );	// Pop the function value off the back of the stack and into the object (-2 == penultimate object on stack) using the key "write" (i.e. method name).
    lua_setglobal( L, "session" );	// Pop the object off the stack into a global named "session".

    and inside the session_write function, you'd retrieve it again like:

    	session*	sessionPtr = (session*) lua_touserdata( L, lua_upvalueindex(1) );

    And now you know all you need to call Lua from C, and have Lua call your C functions back.

  • I want to make an MMORPG…

    A demo session of the eleven chat client

    I hear that's the thing beginning programmers say to game developers. One of those naïve things people want to do who don't know any better. But while I agree it's an illusion to think we, as beginners, and a single person, could just write out the next World of Warcraft, I can totally understand why one would want to do it.

    The reason why MMORPGs are near-impossible to pull off is the same reason that makes them so interesting: They're a big honkin' fun challenge. They combine all the fun tech and its problems. And they're fun to use and well regarded as well. Think about the challenges:

    • Graphics and animation - often even on low-end machines, often in 3D, with bone meshes, inverse kinematics, several detail levels of models and customizable weapons, avatars etc.
    • User interfaces - often completely custom written on top of low-level 3D engines.
    • Maths - figuring out how leveling needs to work, how different weapons interact, but also pathfinding, and graph theory for the NPCs' limited "AI".
    • Social and economic science Managing interaction between players to encourage fun, discourage trolls and griefers, and balance gameplay between beginners and hardcore gamers or at least keep them from interfering with each other's enjoyment. Often also involving trading systems.
    • Networking - Including load-balancing of lots of users, accounting for input lag, streaming level data and video.
    • Security - In addition to traditional hackers, you've got lots of players and script kiddies who are fans and are just trying to game the system a little for personal benefit.
    • Storytelling and content-creation - Non-sequential stories depending on user decisions, and lots of other content overlapping with graphics and animation to keep players coming back, and to bridge the gaps between story missions.
    • Databases - Includes storing all user information with good performance under heavy load and with transactional integrity even on sudden disconnects (e.g. so you don't "pay" for an item and then don't receive it).

    I think the only thing from computing you don't need to know how to do for an MMORPG is how to build hardware dongles and write their drivers.

    And you want to build one?

    Well, yes and no. I wouldn't really want to run an MMORPG. I also already have a job. And I suck at 3D modeling. But a bunch of these problems are intriguing and fun to think about, so I'm tinkering with ideas, writing little test projects that implement this or that part that one would need for such a game. I've also been reading a lot of stuff on Stack Exchange's GameDev sub-section, for example a question on how one calculates the leveling of characters.

    So you might just see me blog a little more about game design in the future. Or this may be the first and only post I'll do about this topic before I lose interest, who knows.

    So what have you made so far?

    Having played a couple hundred hours of Star Trek Online, I realized that a lot of the program felt like a graphical client talking to an IRC server that had a bot that ran the actual game logic. So the networking layer is what I wanted to start with. I'm not sticking to any IRC specification, but simply started implementing a modular chat server that is kinda similar to IRC, and has chatrooms (which are useful for separating maps and their communications overhead), and of course user accounts with blocking mechanisms.

    The advantage of this approach is that, even if I don't ever make a game that uses this, it may be useful to other people who are looking for some sort of networking layer, be it for chat, or for some other social program. And at the very least it let me practice sockets and is allowing me to learn how to use TLS to encrypt a connection.

    Where will it go from there?

    Well, the basic idea is to then build the "bot" that actually implements the game. It will live in chat rooms, one for each map (i.e. a city, planet or whatever) and let you connect to one. Once in a chatroom, it will send you all the data you need to get started.

    In the case of a graphical game that would be models, level maps etc. as well as pending messages you may have (e.g. "crafting operation completed" or "spell prepared" etc., but also remind you of mission rewards you need to accept or present a mission introduction you haven't confirmed as read yet). It will also send status about the player, like inventory, cooldowns of abilities you've recently used etc. and update you about changes in any of these, and inform you of nearby enemies and allies and what they're doing (to you?).

    You can now send commands to this bot, which include character movements, casting a spell, firing a weapon or whatever.

    So, oddly, all of this feels a lot like it can be implemented fairly elegantly and simply as sort of an IRC/e-mail hybrid with read receipts. And this can all be done without needing to have a client, nor any graphics. Heck, you could test this by writing an ASCII client that looks like NetHack.

    The game map

    The game is streamed, so it might not be the best idea to download the entire map at once. Especially if you want to be able to maybe have large or endless maps, or maps that can be re-used for different purposes. You'd want to be able to only load those parts of maps that the user visits for this mission.

    So the obvious approach so far seems to be to just split the map up into tiles that have relative locations to each other and then load the tiles surrounding your player's location, and when the player moves load additional ones so you never run out. This can be a 2D tiled map to start with, but can easily be expanded into cubes that can also be stacked vertically.

    To simplify, I'll probably have only a 2D coordinate inside each tile. The vertical coordinate for a player or NPC inside its particular tile-cube will then simply be decided by the ground level underneath it. Since we can connect the tiles arbitrarily, stairs would still be possible to make. Just create a tile with the stair model at its end:


    The player will seem to walk up the stairs because the ground level rises. If we now connect the left edge of the tile to the tile one level up, the user will easily walk up one level. The only downside of this approach is that a character won't be able to hide under the stairs.

    Another advantage of using such tiles is that they can be used for collision testing. You can just block an entire tile from being stood on.

    That wasn't too hard ... ?

    Well, this is still missing any item management, interaction, mission objective enforcement, combat or pathfinding mechanisms, so it's far from a game, but yeah, it's probably what I will try to implement after the chat server is finished. Or maybe I'll just -- oh look a butterfly.

  • How Drawing on iOS Works

    Someone on Stack Overflow recently asked about the various drawing APIs on iOS, and what the difference between using CALayers directly or using them indirectly through UIViews is, and how CoreGraphics (aka Quartz) fits into the equation. Here is the answer I gave:

    The difference is that UIView and CALayer essentially deal in fixed images. These images are uploaded to the graphics card (if you know OpenGL, think of an image as a texture, and a UIView/CALayer as a polygon showing such a texture). Once an image is on the GPU, it can be drawn very quickly, and even several times, and (with a slight performance penalty) even with varying levels of alpha transparency on top of other images.

    CoreGraphics (or Quartz) is an API for generating images. It takes a pixel buffer (again, think OpenGL texture) and changes individual pixels inside it. This all happens in RAM and on the CPU, and only once Quartz is done, does the image get "flushed" back to the GPU. This round-trip of getting an image from the GPU, changing it, then uploading the whole image (or at least a comparatively large chunk of it) back to the GPU is rather slow. Also, the actual drawing that Quartz does, while really fast for what you are doing, is way slower than what the GPU does.

    That's obvious, considering the GPU is mostly moving around unchanged pixels in big chunks. Quartz does random-access of pixels and shares the CPU with networking, audio etc. Also, if you have several elements that you draw using Quartz at the same time, you have to re-draw all of them when one changes, then upload the whole chunk, while if you change one image and then let UIViews or CALayers paste it onto your other images, you can get away with uploading much smaller amounts of data to the GPU.

    When you don't implement -drawRect:, most views can just be optimized away. They don't contain any pixels, so can't draw anything. Other views, like UIImageView, only draw a UIImage (which, again, is essentially a reference to a texture, which has probably already been loaded onto the GPU). So if you draw the same UIImage 5 times using a UIImageView, it is only uploaded to the GPU once, and then drawn to the display in 5 different locations, saving us time and CPU.

    When you implement -drawRect:, this causes a new image to be created. You then draw into that on the CPU using Quartz. If you draw a UIImage in your drawRect, it likely downloads the image from the GPU, copies it into the image you're drawing to, and once you're done, uploads this second copy of the image back to the graphics card. So you're using twice the GPU memory on the device.

    So the fastest way to draw is usually to keep static content separated from changing content (in separate UIViews/UIView subclasses/CALayers). Load static content as a UIImage and draw it using a UIImageView and put content generated dynamically at runtime in a drawRect. If you have content that gets drawn repeatedly, but by itself doesn't change (I.e. 3 icons that get shown in the same slot to indicate some status) use UIImageView as well.

    One caveat: There is such a thing as having too many UIViews. Particularly transparent areas take a bigger toll on the GPU to draw, because they need to be mixed with other pixels behind them when displayed. This is why you can mark a UIView as "opaque", to indicate to the GPU that it can just obliterate everything behind that image.

    If you have content that is generated dynamically at runtime but stays the same for the duration of the application's lifetime (e.g. a label containing the user name) it may actually make sense to just draw the whole thing once using Quartz, with the text, the button border etc., as part of the background. But that's usually an optimization that's not needed unless the Instruments app tells you differently.