Structure and Interpretation of Computer Programmers

I make it easier and faster for you to write high-quality software.

Saturday, April 28, 2012

Software-ICs and a component marketplace

In the previous post, I was talking about Object-Oriented Programming, an Evolutionary Approach. What follows is a thought experiment based on that.

Chapter 6 of Brad Cox’s book, once he’s finished explaining how ObjC works (and who to buy it from), is concerned with his vision of how Object-Oriented software will be built. He envisions “Software-ICs”—compiled object files defining the code to support a single class (no need for header files, remember) that are distributed with documentation on how to use that class.

Developers or “software librarians” connect ICs together into collections called “categories”, which are implemented as object libraries. It’s a bit unfortunate that “category” is an inappropriate name mainly due to later reuse by NeXT; but then the alternate word “framework” is also unfortunate due to confusion with the computer science term (which allows that AppKit, WebObjects UIKit are frameworks, but Foundation, Quartz and so on are not). But it’s an entirely understandable reuse: in Smalltalk-80 related methods are grouped into categories, and NeXT used the same terminology for a very similar purpose.

Interestingly, Cox allowed for the compiler to generate vtables of selectors for each category, a bit like the Amiga operating system’s library format. That’s to support having different variable types for selectors with the same name. Modern Objective-C doesn’t support that; if you define selectors with the same name but different parameters or return values, you’ll get a warning and your code might not work correctly.

Finally, an application is a network of categories connected by the linker. One (or perhaps more, depending on your design) of the categories in the application contains the app-specific classes.

My reason for bringing this up is that this vision of object-oriented software engineering closely models component-oriented hardware engineering by allowing for software shops to produce catalogs of the components they produce at each level. Just as you can order a single IC, or a circuit board with a few ICs connected, or a whole widget, so you could order a class, or a category, or an application. If you want to build a new application, you might buy a couple of classes from one vendor, a category from another vendor, then write a few classes yourself and integrate the whole lot.

Enough ancient book, talk about the real world.

We have a lot of this, and make quite a lot of use of it. There are loads of Objective-C classes, libraries and frameworks out there for us to use, and to some extent there are catalogs. Many of the components we can use are open source, which means that we can treat the class interfaces themselves as the catalogs. If we’re lucky there’ll be some documentation, perhaps in the form of AppleDoc or a README.

Unfortunately availability vastly outstrips discoverability. You have to go to multiple catalogs to ensure that you’ve exhausted the search space: Google Code, GitHub, BitBucket, SourceForge etc. in addition to finding commercial libraries which won’t be listed in any of those places. Actual code search engines like OpenGrok and Koders are great for finding out about source code, but not so great for discovering it in the first place.

Metacatalogs like Cocoa Objects, Cocoa Controls and CocoaOpen solve part of this problem by letting people list their source code in a single place, but because they’re incomplete they only add to the number of places you need to search.

Then, once you’ve got the component, what do you do? Are you meant to drop the source files into your project? Should you drop the project in and add the library as a dependency of your app? Should you use CocoaPods?

Learn from what we already do

Just as we already push most of the apps we write to a single app store where customers can discover, purchase and install apps in a state where they’re ready to use, we should do the same with components.

[Please bear in mind that like most descriptions of ideas, a lot of nuances and complexity are known but are elided below for the sake of clarity. Comment brownie points will not be awarded for comments that explain how I haven’t considered case X; I probably have.]

A component store would, for browsers, start off very similar to Cox’s idea of a component catalog. You’d go to it and search for a component that suits your needs. You could see a “spec sheet” for each component detailing what it does, what it costs, the terms of using it and that sort of thing. You’d then buy the component if it’s paid for and download it. If the licence permits it you could download the source, too.

The download would drop the binary and headers into a folder that Xcode would recognise as an additional SDK. It would also drop the documentation in docset format into a standard location. An Xcode project would just need to point at the additional SDK and it could pick up all of the components available to the developer.

From the perspective of a component manufacturer, the component store would look a little like iTunes Connect. You’d write your code, then package it up for the store in a standard way along with the description that goes into the “spec sheet”. For open source projects that could just involve git push componentstore master to have the store itself generate the binaries and documentation from the source code.

posted by Graham at 13:50  

Saturday, April 28, 2012

Comparing Objective-C and Objective-C with Objective-C

A while back, I wrote an object-oriented dispatch system for Objective-C. It defines only three things: an object type (the BlockObject), a way to create new objects (the BlockConstructor), and a way to message objects (the dispatch mechanism).

That’s all that the first version of Objective-C defines, as documented in Brad Cox’s Object-Oriented Programming: an Evolutionary Approach.

Objective-C is a hybrid programming language[…]formed by grafting the Smalltalk-80 style of object-oriented programming onto a C language rootstock. Objective-C adds precisely one new data type, the object, to those C provides already, and precisely one new operation, the message expression. Like Smalltalk-80, Objective-C makes no compile-time distinction between different kinds (classes) of objects.

So if you want to try out 1980s-style OOP using Objective-C, the tool to use is not Objective-C itself but BlockObject.

That quote is from the start of Chapter Four (of the Cox edition, not the Cox and Novolbilski edition), which goes on to describe the Objective-C preprocessor, the multiple-dispatch system (using strings as message selectors like BlockObject does—and indeed like Apple’s ObjC does albeit hidden behind the opaque SEL type), and the _msg C function that implements message dispatch.

Factory Objects

Having objects is all very well, but we need some way to create them. In the BlockObject system, there’s a special type called BlockConstructor that configures and returns a new instance of an object.

That’s boring. Objective-C doesn’t define a “constructor type”, can’t we just use objects? Well, yes. Given a class definition like this:

= Stack : Object { id *objects; unsigned int count; }

Objective-C as-was automatically defines a global factory object called Stack that can be used anywhere in the code. You grab it and use it like this:

extern id Stack;

id myStack = [Stack new];
[myStack push: someObject];

From there, Objective-C won’t surprise you much. You can define instance methods:

-push:anObject {
	if (count == MAX_OBJECTS) [self cannotGrow];
	objects[count++] = anObject;
	return self;

and factory methods:

+new {
	id stack = [super new];
	stack->objects = malloc(MAX_OBJECTS * sizeof(id));
	return stack;

Though users of “modern” Objective-C will notice that there’s no compile-time type checking: no Stack *stack variable declaration for example. Indeed there’s no @interface for the class at all; all objects are of type id, you can send any message to any object and what it does with that message is up to the receiver. Just like Smalltalk or Ruby.

posted by Graham at 13:49  

Friday, April 27, 2012

The debugger of royalty

We’ve all got little libraries of code or scripts that help us with debugging. Often these are for logging information in a particular way, or wrapping logs/tests such that they’re only invoked in Debug builds but not in production. Or they clean up your IDE’s brainfarts.

Having created these debug libraries, how are you going to get your production code to use them? Are you really going to sprinkle MyCompanyDebugLog(fmt,…) messages throughout your app?

Introducing step one on the road to sanity: the debug proxy. This is useful when you want to find out how a particular class gets used, e.g. when it provides callbacks that will be invoked by a framework. You can intercept all the messages to the object, and inspect them as you see fit. Here’s the code (written with the assumption that ARC is enabled):


#import <Foundation/Foundation.h>

@interface FZADebugProxy : NSProxy

- (id)initWithTarget: (NSObject *)aTarget;



#import "FZADebugProxy.h"

@implementation FZADebugProxy {
    NSObject *target;

- (id)initWithTarget:(NSObject *)aTarget {
    target = aTarget;
    return self;

- (NSMethodSignature *)methodSignatureForSelector:(SEL)sel {
    NSMethodSignature *signature = [target methodSignatureForSelector: sel];
    if (signature == nil) {
        signature = [super methodSignatureForSelector: sel];
    return signature;

- (BOOL)respondsToSelector:(SEL)aSelector {
    return [target respondsToSelector: aSelector] ? YES : [super respondsToSelector: aSelector];

- (void)forwardInvocation:(NSInvocation *)invocation { = target;
    SEL aSelector = [invocation selector];
    [invocation invoke];


And no, there isn’t a bug in the -initWithTarget: method. The slightly clumsy extraction of the selector in -forwardInvocation: is done to avoid a common problem with using Objective-C inside the debugger where it decides it doesn’t know the return type of objc_msgSend() and refuses to call the method.

You would use it like this. Here, I’ve modified the app delegate from BrowseOverflow to use a proxy object for the object configuration – a sort of domain-specific IoC container.

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
    BrowseOverflowViewController *firstViewController = [[BrowseOverflowViewController alloc] initWithNibName: nil bundle: nil];
    firstViewController.objectConfiguration = (BrowseOverflowObjectConfiguration *)[[FZADebugProxy alloc] initWithTarget: [[BrowseOverflowObjectConfiguration alloc] init]];
    TopicTableDataSource *dataSource = [[TopicTableDataSource alloc] init];
    [dataSource setTopics: [self topics]];
    firstViewController.dataSource = dataSource;
    self.navigationController.viewControllers = [NSArray arrayWithObject: firstViewController];
    self.window.rootViewController = self.navigationController;
    [self.window makeKeyAndVisible];
    return YES;

The bold line is the important change. The cast silences the compiler’s strict type-checking when it comes to property assignment, because it doesn’t believe that NSProxy is of the correct type. Remember this is only debug code that you’re not going to commit: you could switch to a plain old setter, suppress the warning using diagnostic pragmas or do whatever you want here.

At this point, it’s worth running the unit tests and using the app to convince yourself that the behaviour hasn’t changed at all.

So, how do you use it? Shouldn’t there be an NSLog() or something in the proxy class so you can see when the target’s messaged?


Step two on the road to sanity is to avoid printf()-based debugging in all of its forms. What you want to do here is to use Xcode’s debugger actions so that you don’t hard-code your debugging inspection capabilities into your source code.

Set a breakpoint in -[FZADebugProxy forwardInvocation:]. This breakpoint will be met whenever the target object is messaged. Now right-click on the breakpoint marker in the Xcode source editor’s gutter and choose “Edit Breakpoint…” to bring up this popover.

Xcode's breakpoint popover.

In this case, I’ve set the breakpoint to log the selector that was invoked, and crucially to continue after evaluation so that my app doesn’t stop in the debugger every time the target object is messaged. After a bit of a play with the app in the simulator, the debug log looks like this:

GNU gdb 6.3.50-20050815 (Apple version gdb-1752) (Sat Jan 28 03:02:46 UTC 2012)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin".sharedlibrary apply-load-rules all
Attaching to process 3898.
Pending breakpoint 1 - ""FZADebugProxy.m":37" resolved
Current language:  auto; currently objective-c
0x10271:	 "stackOverflowManager"
0x102a0:	 "avatarStore"
0x10271:	 "stackOverflowManager"
0x10271:	 "stackOverflowManager"
0x102a0:	 "avatarStore"

Pretty nifty, yes? You can do a lot with Xcode’s breakpoint actions: running a shell script or an AppleScript are interesting options (you could have Xcode send you an iMessage every time it send your target an Objective-C message). Speaking out the names of selectors is fun for a very short while, but not overly useful.

Xcode’s breakpoint actions give you a much more powerful debugging capability than NSLog(). By using breakpoint actions on an Objective-C proxy object, you can create highly customisable aspect-oriented techniques for debugging your code.

posted by Graham at 18:53  

Thursday, April 19, 2012

TDD and crypto in one place

Well, I suppose if I’ve written two books, it’s about time I wrote a contorted blog post that references both of the worlds.

I recently wrote an encryption module for an app, and thought it’d be useful to share something about the design process. Notice that the source code here was quickly thrown together for the purposes of demonstrating the design technique, and bears little resemblance to the code I was actually writing (which was in a different language). A lot of the complexity I was actually trying to deal with has been elided in order to bring the story to the fore.

Steps 1a, 2a, 1b, 2b etc: Design the API hand-in-hand with a fake implementation.

A pass-thru implementation would work here, but I prefer to use something that actually does change its input just so that it’s clear that there’s a requirement the input should be acted on. You might end up with tests that look something like this:

@implementation CryptoDesignTests
    id <StringEncryptor>cryptor;

- (void)setUp {
    cryptor = [[ROT13StringEncryptor alloc] init];

- (void)tearDown {
    cryptor = nil;

- (void)testEncryptionOfPlainText {
    NSString *plainText = @"hello";
    NSString *cipherText = [cryptor encipherString: plainText];
    STAssertEqualObjects(@"uryyb", cipherText, @"This method should encrypt its parameter");

- (void)testDecryptionOfCipherText {
    NSString *cipherText = @"uryyb";
    NSString *plainText = [cryptor decipherString: cipherText];
    STAssertEqualObjects(@"hello", plainText, @"This method should decrypt its parameter");


Where the protocol looks like this:

@protocol StringEncryptor <NSObject>

- (NSString *)encipherString: (NSString *)plainText;
- (NSString *)decipherString: (NSString *)cipherText;


Of course you can implement this however you want. Here’s one potential implementation (which may have its problems, but is only being used to guide the API design):

@interface ROT13StringEncryptor : NSObject <StringEncryptor>


@implementation ROT13StringEncryptor

- (NSString *)rot13: (NSString *)original {
    NSMutableData *originalBytes = [[original dataUsingEncoding: NSASCIIStringEncoding] mutableCopy];
    for (NSInteger i = 0; i < [originalBytes length]; i++) {
        NSRange cursor = NSMakeRange(i, 1);
        char c;
        [originalBytes getBytes: &c range: cursor];
        if       (c >= 'a' && c <= 'm') c += 13;
        else if  (c >= 'n' && c <= 'z') c -= 13;
        else if  (c >= 'A' && c <= 'M') c += 13;
        else if  (c >= 'A' && c <= 'Z') c -= 13;
        [originalBytes replaceBytesInRange: cursor withBytes: &c];
    return [NSString stringWithCString: [originalBytes bytes] encoding: NSASCIIStringEncoding];

- (NSString *)encipherString:(NSString *)plainText {
    return [self rot13: plainText];

- (NSString *)decipherString:(NSString *)cipherText {
    return [self rot13: cipherText];


Step 3. Write an integration test.

Having designed the API for the class, I'd like to write an integration test that does whatever it is that the app is trying to achieve. In my real app this task was that two cryptors hooked up to each other should agree on a key and pass a message between each other. For this demo, we'll require that the cryptor can round-trip a message and end up with the original text.

@implementation CryptoRoundTripTests
    id <StringEncryptor>> cryptor;

- (void)setUp {
    cryptor = [[ROT13StringEncryptor alloc] init];

- (void)tearDown {
    cryptor = nil;

- (void)testRoundTripEncryption {
    NSString *plain = @"Mary had a little lamb.";
    NSString *outOfTheSausageMachine = [cryptor decipherString: [cryptor encipherString: plain]];
    STAssertEqualObjects(plain, outOfTheSausageMachine, @"Content should survive an encryption round-trip");


This test passes without any extra work.

Step 4. Give that lot to the poor sap who needs to write the app.

That's right, the other developers can start working straight away. They've got the protocol which tells them what the object can do, the unit tests which explain how the API should work and the integration test that explains how you expect the object to be used.

Leaving you free to:

Step 5. Point the integration test at a different implementation of the protocol.

But you haven't written that implementation yet! It must be time to:

Step 6. Write it.

You know when it works because you have integration tests. You know it'll work with the app the other person's writing because they've got the same integration test.

So the extra artefacts that some people see as a wasteful byproduct of code-level testing — the fake implementation of the protocol for example — are demonstrably useful in this scenario. Other developers on the team can use the fake implementation as a stand-in while you're head-down coding up the production code. It's also much easier to design a class when you're not also sweating over how the internal details are going to work out, so you probably get where you're going quicker too.

posted by Graham at 21:43  

Sunday, April 15, 2012

Culture, heritage and apps

I said earlier on Twitter that I’m disappointed with the state of apps produced for museums and libraries. I’d better explain what I mean. Here’s what I said:

Disappointed to find that many museum apps (British Library, Bodleian, Concorde etc) are just the same app with different content. :-(

In each case (particularly Concorde) there’s some magic specific to the subject. Rubber-stamping the app doesn’t capture that magic.

They’re all made with a tool called Toura that sits atop PhoneGap. Makes it easy to get the content into the apps at the cost of expression.

So to be clear: my problem is not with the museums and other heritage sites. I’m familiar with the financial and political problems associated with running a museum. My social circle includes curators, librarians and medieval manuscript experts and I know that money is tight, that oversight is close and that any form of expenditure in promotion must result in a demonstrable increase in feet through the door and donations to be considered a success.

The Magna Carta

My problem is also not with the Toura product and the team behind it. They’re to be commended for identifying that while the culture and heritage community doesn’t have much money, they still deserve to be represented in our tablets and phones. The apps I listed above all feature astounding objects: examples include the Concorde aircraft, the Lindisfarne gospels and the Magna Carta.


So why are these apps disappointing? It’s basically because they’re all the same. Each of these objects has its own magic: the unique shape of Concorde, the vivid colours of the gospels and the constitution-defining text of the Carta. So why present them in the same way? Why not make a Concorde app that evokes aerodynamic speed, an illuminated Lindisfarne gospels app, and a revolutionary Magna Carta app? As many people have explained, it’s because standard content-viewing apps like Toura are all these institutions can afford.

Folio 27r from the Lindisfarne Gospels: image from Wikimedia Commons

There’s something seriously wrong in our app industry. We’ve created a world in which apps are too cheap, and developers struggle to make a living from 70¢ per sale. Simultaneously, we’ve created a world in which apps are too expensive: the keepers of some of the world’s most interesting objects can’t afford to showcase these with more than a generic master/detail view. What’s that about?

What mindset leads us to demand high worth, when we’re making products that we can’t convince customers contain any value? If they don’t see the value, are we deluding ourselves? People talk about events like the Instagram acquisition as being evidence of a bubble subset of the app economy. But isn’t that oxymoron – a free app that’s worth a billion dollars – just an exaggerated version of what the rest of the industry is up to?

Let me present this problem in a different way, by moving from apps about museums to museums about apps.

The Rijksmuseum, Amsterdam.

Last month I spent an enjoyable time with some very good friends in Amsterdam, the city of both apps and museums. While walking around the Rijksmuseum, I reminded myself that we’re in the middle of the possibly the largest and definitely the fastest technology-mediated social revolution humanity has ever experienced. Those of us enabling the application of this technology are literally providing the fulcrum around which our species is revolving.

During this year, museums will look at apps as cultural, historical and ethnological artefacts in their own right. Over the coming century, heritage institutions will shape the way that our brief period in time is presented for posterity. Be sure that all of the million apps currently available across all app stores will not be on display.

As with any presentation, perfection is achieved not when there is nothing left to add but when there is nothing left to take away. Just as you can’t (and wouldn’t want to) walk into a museum and see every manuscript from the 10th century on display, you won’t be able to walk into the Nieuwerijksmuseum in 3112 and see every app from the 21st century. Only those that are considered cultural treasures will be given pride of place in the galleries. Can something that apparently has no value be considered a cultural treasure? Will any of your apps be part of the presentation? I don’t think I’ve yet produced anything that will be, and that needs to change.

posted by Graham at 22:00  

Thursday, April 12, 2012

Test-Driven iOS Development

Here it is, after more than a year in the making, the book that they really did want you to read! Test-driven IOS Development (Developer’s Library) (affiliate link) has finally hit the stores[*].

I wrote this book for the simple reason that it didn’t exist. Like Professional Cocoa Application Security (Wrox Professional Guides) (another affiliate link), I knew that the topic was something many people were interested in, including myself. I wanted a single place I could go to find out about using Xcode for writing unit tests, and how the Test-Driven Development approach could be applied to writing Cocoa code with Objective-C.

Well, it turned out that this place didn’t exist, so I tried my best to create it. In the book you’ll see how the BrowseOverflow app was built entirely in a test-first fashion, using OCUnit and Xcode 4. Along the way, I hope you’ll see how TDD can be useful for your own projects.

If you’re interested in finding out more about test-driven development for iOS apps, I’ll be giving a talk on the subject at iOSDev UK in Aberystwyth in July. And of course I’m happy to field questions about the book or about TDD here or over on Twitter. And of course my unit testing video training course is still available from

Happy testing!

[*]The availability appears to vary by country and format, so if it isn’t in your favourite (e)bookstore yet just be patient and keep refreshing that page like it’s the WWDC site.

posted by Graham at 23:01  

Thursday, April 12, 2012

On the magic of key agreement

Imagine that you want to implement AirDrop, or something like it. Two computers that have (possibly) never communicated before are going to share a file. Now you know that you want to encrypt the file in transit so that only these two computers get to see the file, but how do you set up an encryption key? You can’t generate it on one of the computers and send it over, because that way lies the infinite stack of turtles: how do you keep the key secure in transit?

The solution (or at least, one solution known as the Diffie–Hellman protocol) turns out to be beautifully simple. First, arrange that everyone knows two numbers, which we’ll call the base or generator g and the modulus m. For example, in the SKIP protocol these two numbers are documented:

g = 0x cf ca d9 90 8d 98 c1 c4 4e 4e 88 20 69 e8 7b fb
d0 ca e1 ee 24 64 a9 6c 89 70 a0 8c 86 00 30 c6
c2 83 4a 0d 82 bd a5 ba 10 cf 80 af 61 96 11 4b
3b 87 a9 7e 59 98 c4 aa 0c bd b7 2b 23 18 33 64
f3 b6 2e 1a 4d e8 86 85 aa de 78 a1 5d ce 65 33
75 7a 9d 00 b9 9e 05 26 ed 79 62 15 97 c4 06 26
fa 51 e1 5e f5 1d cb d2 23 e2 73 e5 f2 c7 3c c4
de 58 cd 3b b6 15 3c aa 84 7c fa 5f cb 6b 8d 78

m = 0x dd c9 08 3e 8f ae ef 28 16 ad 50 a9 68 ac bd 04
8e 90 9e ab 5d 41 fc 0a 51 3f 86 0a c4 1b 22 b9
30 dc a3 0a 59 73 05 38 59 b7 85 66 df ff c6 5b
b9 1f fe 44 d9 d6 5e cb 9b 68 38 a1 fd 25 3f 01
51 88 9e 93 c3 22 24 f9 03 e6 9b 8b 07 34 9d 9f
9b 38 4b c0 97 03 dc e5 2e 92 47 4c 2c e1 59 26
14 82 49 dd 58 13 91 05 12 11 a1 45 06 ac 11 8f
b1 83 53 46 93 88 9a 46 b1 8a 01 50 cb 5e 82 55

The two computers each generate a random number, which we’ll call x_A and x_B. They keep these numbers secret, but can independently calculate:

r = g^(x_A) mod m

s = g^(x_B) mod m

and they send each other r and s. The first computer can use x_A and s to derive:

k_A = s^(x_A) mod m = g^(x_A.x_B) mod m

while the second uses x_B and r to derive:

k_B = r^(x_B) mod m = g^(x_A.x_B) mod m ≡ k_A

They got the same key! Because you can’t gain any of the secrets k, x_A or x_B knowing only r and/or s, the two systems can negotiate a secret key that’s only known to the two of them without having to first establish a secure channel. I just think that’s really cool.

If you want to use this stuff in an iOS app, OpenSSL contains an implementation of Diffie–Hellman. On Android javax.crypto.KeyAgreement can do it for you.

posted by Graham at 19:24  

Wednesday, April 4, 2012

On my newer competence

This time last year, I evaluated myself against the programmer competency matrix. So where am I one turn around the daystar later? I have to admit that this was mainly because I was jet lagged in a hotel room in Seattle and had nothing else to do last year :-), but nonetheless it’s time to reflect on how things have gone.

Last year’s goals

I wrote:

it looks like my priorities for research goals over the upcoming year or so should be:

  1. Improve breadth and depth of API knowledge
  2. Investigate systems decomposition of a smartphone/desktop/internet system.
  3. Learn how to create and support DSLs
  4. Write and publish improved developer support tools

So how did that go?

  1. I improved here. I didn’t have a project for doing this, I just used the APIs some more. Things that I previously remembered but needed to look up I now automatically get right, and some things I didn’t know about I now do. There’ll always be room for more.
  2. My work on GNUstep web is only part of this goal.
  3. I haven’t done anything with DSLs, as I haven’t found a need for them in the last year. Maybe that’s because I don’t understand where they can be used :). Part of my GNUstep web project will include rules engines a la Direct 2 Web, so I guess I’ll have to tackle this soon.
  4. Again, in hand with the GSW project. I’ve already published a code generation tool. Some of the things I learned about at NSConference were very inspirational.

What about this year?

I’ll just dive into the matrix. If nothing’s changed from last year you won’t see any comments.

Computer Science

  • Data Structures: still level 1. I’ve learned a little more about level 2 aspects.
  • Algorithms: level 2.
  • Systems programming: level 3, if you’re willing to accept that block-diagram-level knowledge counts as knowledge on the level chart :)

Software Engineering

  • Source Control: level 3.
  • Build Automation: level 3.
  • Automated Testing: level 3.


  • Problem Decomposition: level 3.
  • Systems Decomposition: level 3. As I said earlier, I made this a specific goal of the last year, and I believe it’s paid off.
  • Communication: level 3. I’ve also done what I set out to do last year, and dialled back on the commitments to conferences and books.
  • Code Organisation Within a File: level 3. As I said last year, this is subjective.
  • Code Organisation Across Files: level 3.
  • Source Tree Organisation: level 3. For these two I’ve decided it’s what you see in the IDE (Xcode, AppCode and Eclipse are my regular haunts) that really matters, though I try to make the filesystem layout match that.
  • Code Readability: level 3. I’m not quite that bastard who rejects code because it doesn’t conform to the standard; though I used to be :-(
  • Defensive Coding: level 2. Still don’t have a “library” for doing that (I’ve written an iOS security library that’s internal to O2, but that’s not what this is about.)
  • Error Handling: level 3 (with s/xception/rror/, because we appreciate that error-handling should be sympathetic to the platform/framework style, just like other code.)
  • IDE: level 3. It was mainly working with AppCode that spurred me into investigating ways to automate IDE functions.
  • API: still going to claim level 1, because I’d say it’s for other people to decide whether they think I have sufficiently “vast and in-depth knowledge”.
  • Frameworks: level 2.
  • Requirements: level 3.
  • Scripting: level 3 (though this improvement only just scraped in this year.)
  • Database: level 1. I’m starting to see more projects where a better knowledge of databases is required, if one of those lands in front of me I’d better improve at knowing databases. However as I said last year it’s the kind of thing where I can use an ORM to get it mostly correct, then investigate whether any issues remain that require deeper inspection.
  • Experience

    I skipped this section last year, but just for kicks I’ll go through it this time. No comparisons with last year, and if you can’t work out why you’re not allowed higher than Level 0 on any entry in the matrix ;-).

    • Languages with Professional Experience: level 2. Functional programming I have worked with includes LISP, C++ and Java (yes, that Java).
    • Platforms with Professional Experience: heh, this depends as I said in a recent talk on how you define “Unix”. Conservatively I could say Unix, Mac, iOS and Java: level 1.
    • Years of Professional Experience: not that I think this matters, but level 2.
    • Domain Knowledge: wait, which domain? In the world of telco (where I currently spend my time) I’m a complete noob. In fact I’ve spent most of my career as a complete noob. Whether that’s good or bad, and something I should or would want to change, is open for debate. Maybe that’s another blog post, later.

    Yeah, having written that section I now understand why I left it out last time.


    • Tool Knowledge: still level 2. I’ve written and published some codeine scripts, not sure it’s enough to qualify at level 3.
    • Languages Exposed to: still level 2. I did some concurrent programming in Objective-C, but no Erlang, Oz or Prolog. OK, I have done some Prolog tutorials, and Oz is the operating system on my Cambridge Z88 but I’m going to assume that it’s a different Oz.
    • Codebase knowledge: hard to gauge. We have about 100 repos on my team, some I’m not working on, some I collaborate on, some I’m the sole author. We’ll call this one unknowable for now.
    • Knowledge of Upcoming Technologies: level 3.
  • Platform Internals: level 2.
  • Books: still level 1, but I’ve written two now. I don’t imagine writing any more, there must be a better way to share knowledge.
  • Blogs: level 3.
  • This year’s goals

    Some of the things I planned for last year still need to be done:

    1. DSLs
    2. Developer tools

    To which I’ll add some things that, probably, don’t directly follow from the competency matrix, but which are nonetheless goals of mine.

    1. More do, less talk. A side-effect of being “the security guy” in my corner of the industry is that I do a lot of talking about security in other peoples’ products, and not much making products. It’s time to find a product I believe in, and own the crap out of it. That means investing my time in it long-term, not diving in and out like a consultant. Consultants are great and consultancy is great: but I want to build something.
    2. Mentor new developers. A recurring theme from my discussions in Appsterdam and at NSConf is that while I can certainly do a lot to become a better programmer, that won’t scale as well as making a load of other people better programmers. If I can help at least one person become a better developer than I am, I’ll improve my own abilities along the way and then there’ll be two people better than I am.

    That will, I think, do for now.

    posted by Graham at 12:00  

    Sunday, April 1, 2012

    An apology and an opportunity

    Today’s earlier post, UX is snake-oil bullshit, was indeed an April Fool. Sorry to the people who had their “WTF blood boil”, among other reactions.

    I’m also sorry to the people I parodied in the post. Please feel comfortable knowing that if I didn’t respect you and what you did, I wouldn’t feel willing to send you up in that fashion.

    Now of course some people saw the joke, and some people didn’t. But what was most interesting was that in both camps there was a (large) fraction of respondents who recognised in the jokes the germ of truth. They realised that the humour came not from eschewing reality completely, but from taking cues from it and dialling them up to 11 so that the absurdity became clear.

    So each of the points in my previous post is true, just not as true as the rhetoric would have you believe. The field of UX does contain people who can spout the platitudes, but don’t back it up. There is an amount of “look at this fail”. It is a fast-moving field, so it’s hard to find timeless, consistent, actionable advice. And there are people who will tell you that whatever they’re doing (or selling) is UX, that when bolted to your product creates magical UX goodness.

    What that means is not, as my April Fool post concluded, that it’s time to withdraw from study of User Experience. It means that there’s a tremendous appetite among software engineers for resources offering specific, actionable tasks that we can undertake to improve the experience of our products.

    One of the books on user experience I’ve got the most out of is Don’t Make Me Think!: A Common Sense Approach to Web Usability by Steve Krug. When he does have some specific issue to point out in the design of a website, he also shows either how the problem was or could be addressed. We could do with a “Don’t make me think!” for the mobile user experience. We need someone to cherry-pick the research on user testing, ethnography and other techniques, telling us what’s important and what to ignore. What we can be doing, and what we should be doing.

    That’s a great opportunity for someone who wants to become the Smartphone UX Boffin.

    posted by Graham at 16:56  

    Sunday, April 1, 2012

    UX is snake-oil bullshit

    There, I said it. I feel better already. There are people in the world who’ll tell you that the most important thing in the world is UX, that if your software isn’t UX-compliant it isn’t worth shit. Here’s why that’s wrong.


    “Consider the user.” “The user is king.” Yeah? Well why do you name your vaunted “user” after a fucking drug addict? You’re lying, because you know no other life. The user is the poor schmuck that you’re trying to push your apps on, just like a crack dealer. Only of course you don’t call it “crack dealing”, you call it “freemium” and act like it’s some kind of fucking service to let your crackheads – sorry, users – “get familiar” with the product before you start bleeding their wallets.

    Besides, what does it even mean? Consider the user. I looked at my software, and I think users should be grateful that I let them even fucking touch the app store page with their filthy uneducated hands. There, I’ve “considered” the cretins, so I’m doing UX right am I?

    The only examples are bad examples

    Ever seen a UX talk at a software dev conference? I’d be surprised if you haven’t, because these vultures who call themselves “UX consultants” are actually professional speakers who love nothing better than to tell us hard-working developers how we do everything wrong. You’ll be able to spot them if you’re at a conference, they’re the people with the designer jeans and the designer wine glasses and the fucking designer nostril hair. They’ll open their talk with some rant about how the starving children in Africa can wait for help until someone’s fixed the way the fucking tap works in the shower of their six-star hotel room they were in when they gave the same talk in Dubai or some other place us lowlife software engineers can’t afford to visit. You’ll get the impression that we’re the Morlocks, and this fucking Eloi has deigned to come down from the surface to remind us how much better life is up there, and only wants £2000+expenses to do it.

    And then what’s the talk about? It’s a series of examples of what they say is really bad UX, that we’re supposed to sit there and laugh at with them? Fuck that shit. Some colleague of mine has been sitting in a cubicle busting a gut to produce this piece of software, probably with some manager riding their ass and a bunch of conflicting requirements coming from the douchebags in marketing and sales and business analysis and all those other people who don’t know how the fuck a computer works but think their opinions are somehow valid. And this engineer somehow performs the miracle of reconciling all these different inputs, making everybody happy and at the same time writing this gnarly piece of software. You want me to laugh at that? Whoever that is should be given a medal.

    The best bit is of course that these examples are completely worthless. My product doesn’t make people choose which of the fifty states they’re in even when they don’t live in the US. It isn’t some bridge somewhere that projects the cock-signal onto itself at sunset. It isn’t some medical device that kills patients when the nurse forgets to press a button. That means I’m doing better than all the examples, which means I’m doing UX right. Right?


    If you manage to get past the fluff talk and the bad examples and trap a UX person into asking what a good example is, you’ll get a different answer every sodding week. Last week they were all telling us how interfaces should be discoverable and how you should have UI elements with clear actions like buttons and things. This week it’s all about these completely arcane and undiscoverable gestures; I mean what the fuck? The last twitter app I downloaded made me page through about ten pages of user manual before it even let me send a fucking tweet, which is the whole point of the piece of crap. You’re supposed to swipe right with three fingers making the shape of the Eye of Horus or some bullshit to retweet, or something like that. The fact is I didn’t read the manual (sorry, “soft landing” as the hipsters want me to call it) so I’ve got no fucking clue how to do even the simplest of things.

    And that’s just this week’s UX hotness. Next week it’ll all be Jordi LaForge visors or some crap, because that went really well for that Virtuality company back in the 1990s.

    Which brings me on to the next way to win at UX without actually doing any work. Build whatever shitty UI you want, and just wait for the UX consultant circlejerk to decide that the way you did it is the way all apps should do it. You’ll only need to wait a fortnight, maybe a month tops.

    Everything – or anything – is UX

    The best thing is that you too can be in on this party! Whatever you know now, whatever you’re an expert in, you can claim is part of the user experience. You know OpenCL? That’s about making things fast and responsive, therefore you’re a UX consultant! You know Core Audio? Making machines go “ping” when something goes wrong is part of the user experience! You’re a project manager? Fuck that, you’re a User Experience Coordinator!

    And so the final way in which I’m winning at UX without really trying is that whatever the fuck it is I do is user experience. Nobody wants to be hacked, right? So security contributes to the….experience…of the…crack addictuser, right?

    Enough of this crap.

    It’s time to acknowledge that UX is a complete load of snake oil, and that its biggest contribution to society has been to reduce unemployment among people who think £150 is a modest amount to pay for a shirt. It’s time to show that us engineers can make software without their help, just as we did both before they came along and indeed while they were swanning around being better than us.

    Therefore I introduce my latest initiative, the Clueless Losers are Inexperienced paradigm for software design. CLI harks back to the days when we knew that computers are tricky and software is hard, and we didn’t apologise for it or pay consultants to apologise for us. It acknowledges that software is hard to write, so it’d sure as hell be hard to use and you should all be damned grateful if we allow you to use it at all.

    There won’t be expensive hands-on labs or conference talks about how to make CLI apps. Just do whatever it is you need to get the software working. We acknowledge that people are inexperienced and clueless when it comes to software, so there’s no point going out of our way to make things easier for them because it just makes it easier for them to mess it up.

    Oh, and I fixed that crack pusher bullshit that’s been going on. We’re not going to call people “users” like they’re some farm of addicts waiting for their next hit. The word “loser” doesn’t have those pejorative connotations, so in CLI that’s what we call people who interact with our software.

    I’m really excited about the CLI, and about heralding the start of the post-UX-bubble software economy. I hope you’ll all join me in making software as complicated as it deserves to be.

    posted by Graham at 10:33  

    Powered by WordPress