Coding. Standards.

I just realised that this month marks the 10th anniversary of my first payment for writing software (on, of all the weird things to be writing software on in 2002, a NeXTstation)! What have I learned from those ten years? What advice would I give to someone who wants to do this stuff for at least 10 years?

Programming is the easy bit.

Well, comparatively. There are hard bits in programming, and every few years a new paradigm comes along that means you have to unlearn whatever it was you were doing and learn something else to do anyway. So learning programming is never done, but still programming is easier than:

  • estimating. The one project I’ve worked on that finished on its planned completion date only did so by accident.
  • getting any kind of agreement out of two or more people.
  • accepting that the other person isn’t a dick, but has different goals and problems than you.
  • objectively evaluating your own work.
  • objectively assessing someone else’s evaluation of your work.
  • stopping programming when you’re done.

You always need to be learning.

You can’t compete on price in the software market, because there’s always some student somewhere who’s willing to do the same work for free. It was Mike who first taught me that. You have to compete on quality, which means you need to strive to improve your own quality. Because other people are too, so you need to run just to stand still.

There are various ways to learn, and they’re not mutually exclusive. A combination of books/articles, experimentation, and discussion with peers is valuable. If your town has an Appsterdam or a CocoaHeads, get along and say hello.

You probably don’t want to be doing this in 10 years.

I was actually a UNIX programmer a decade ago (well, I was mainly a student). Then I was a barman, then sysadmin, then a Linux server application programmer, then a Mac app programmer, then a contract Mac programmer, then a Java app programmer, then a security consultant, then an iOS app programmer. There’s only a small probability that I’ll be an iOS app programmer in 10 years.

This world moves really quickly. Ten years ago, the iPod was a new and relatively risky proposition. Macs used PowerPC CPUs. Windows XP was the new hotness, and .NET was just about to appear – meanwhile Mac OS X was a sluggish amalgam of NeXT, Java and legacy code. Java, by the way, was run by a now-defunct company called Sun Microsystems, which was trying to work out how to survive the dot-com crash.

Speaking of the dot-com crash, it seems highly likely that within the next decade we’ll see the dot-app crash. App downloads are worth $0.18¢ each, but an app costs $200k – apparently it’s hard work. That means you’ve got to either get yourself into the long tail value-wise (i.e. have a very good app that people will pay for), or you’ve got to find a million users for version 1.0.

For everyone else, the market isn’t worth staying in long-term. The market will bore of brochureware apps, only a few high-value brands will be able to support unprofitable vanity apps, and VCs will realise that throwing their money after an app with no profit strategy is the same as throwing their money after a website with no profit strategy.

It’s likely that at least one of the companies that’s big in the current software world – Microsoft, Apple, Oracle, Google and the like – will be big in the software world of 2022. It’s also likely that there’ll be some new comers that change things completely: Facebook and Twitter didn’t exist ten years ago, and neither did Android, Inc. Sometimes companies that seem to be in an interminable tailspin – like Apple – turn themselves around and become successful.

Learn more than one thing

This is related, in part, to what came above: the thing you’re using right now may not exist, or may be hard to get work in, in a few years’ time. On the other hand, some things seem to outlive the cockroaches: C – and by extension, languages that can link somewhat seamlessly with C like C++, Fortran and so on – have been going on forever. It can be hard to predict which of these camps your favourite tech sits in, so learning more than one thing keeps you employable.

More than that, if your technology of choice comes from a single supplier (e.g. Microsoft, Apple, Embarcadero) then diversification just makes good business sense. This particularly applies in the age of the app store where that sole supplier can also be your sole vendor – you don’t want to sign your entire business’s value over to one other company.

Learning another thing makes you better at the first thing

This is another reason why diversifying your technology portfolio is beneficial. Many of the changes I’ve made recently in the way I write object-oriented software come from talking to Clojure programmers.

The more different things you know, the more connections you’ll be able to make between them. The more you’ll be able to critically analyse one technology, beyond what the vendor tells you. And the more you’ll be able to understand other new things and incorporate them into your Weltanschauung.

Conclusion

My summary could be “learn whatever you can: you never know which bits you need”. Or it could be “don’t rely on your supplier to solve all of your problems”.

I think it’s actually going to be: analyse everything. Reflect on your work: what went well? What didn’t? Could you have done things better? If you don’t think you could have, then you’re probably wrong: what would you need to know to identify the bit that actually could’ve gone better?

But know when to stop, too. Analysis paralysis is as much of a problem as going in blind. At some point, you need to suck it up and move on. Trading these two things against each other is the real difficulty in software engineering.

Objective-C literals and subscripts

If you’re using clang from their website instead of sticking with Apple’s release, you get support for Objective-C literals and object subscripting. I thought I’d take the BrowseOverflow app and apply this new syntax to it. Notice that the code below doesn’t match what’s in github, which still works with currently-released versions of Xcode and their compilers.

Indexed/Keyed subscripting: we’ve seen this before

Using the syntax described above, you can subscript into an object using something that looks like the traditional C square bracket notation. If you use an integer, you get indexed subscripting:

        Answer *thisAnswer = question.answers[indexPath.row];

If you use an object, you get keyed subscripting:

            NSMutableDictionary *userInfo = [NSMutableDictionary dictionaryWithCapacity: 1];
            if (localError != nil) {
                userInfo[NSUnderlyingErrorKey] = localError;
            }

This is something that’s been available in many languages before. Smalltalk (from which Objective-C derives) had a well-defined subscripting syntax across all objects, using at: and set:at:. C++ permits classes to supply the operator[]() method to use C-style index subscripting just as Objective-C does.

An aside in defence of operator overloading

If you look at traditional Objective-C syntax, you can see that (roughly speaking, and anyone who points out the edge case isn’t my friend) there are the things in square brackets that are objects and messages, and the things without square brackets are primitive types. There are two different worlds, and never the twain shall meet.

But actually, we’ve learned that encapsulation is good, and that allowing people to concisely express their intent is better than making them deal with our implementation details. Therefore we want to integrate our data types in the language. We want to tell people “add this thing to the other thing”, not “you need to call this function with these parameters which will add the things”.

Providing custom implementations of the standard operators is the best way of doing that. Yes, it hides what’s happening: that’s the point. Yes, it can be abused: the entire software industry is based on a foundation that makes it possible to write bad software. If you want to take away tools that can be used to introduce bugs, you need to take away everyone’s compilers and interpreters.

An aside on the aside about Objective-C operator overloading

So far, Objective-C objects can provide custom implementations of two C operators: the field access operator . used for type safe property access, and the subscript operator [] used as we saw before I digressed.

The reason subscript overloading works in ObjC is that it’s illegal to apply arithmetic operations to object pointers. In C, foo[bar] is just a fancy way of writing *(foo+bar) (which is why bar[foo] also works). If you’re not allowed to apply the [] operator to an id, then you know something else must be happening: i.e. you know that the object subscript behaviour is required.

Well the fact that you can’t do pointer arithmetic on an id means that you could also, for example, check for illegal use of the + operator and call -objectByAddingObject:.

Back to the point: boxing done well.

Many languages make some attempt to “box” primitive types like numbers in high-level value types like number objects. This often causes problems: for example in Java, which has both automatic boxing and method overloading, I could do this:

public void foo(int x) { … }
public void foo(Java.lang.Integer x) {…}

foo(3);

It’s not clear whether 3 refers to the primitive type or to the object type, and therefore it’s not clear which method will get called.

The same problem could have been encountered in ObjC: does foo[3] refer to indexed subscripting or to keyed subscripting using an NSNumber instance?

Thankfully whoever designed the Objective-C boxing behaviour decided it must always be explicit. You can get a number like this:

- (void)browseOverflowViewControllerTests_viewDidAppear: (BOOL)animated {
    NSNumber *parameter = @(animated);
    objc_setAssociatedObject(self, viewDidAppearKey, parameter, OBJC_ASSOCIATION_RETAIN);
}

but otherwise numbers will always be treated as numbers, not as objects.

When it all gets a bit much

You’ve got two seconds, the house is on fire, what does this line do?

    NSString *questionBody = [parsedObject[@"questions"] lastObject][@"body"];

It can be a bit hard to read that, and to pick out which brackets go with messages and which go with subscripts. I can imagine the sort of people who like to issue pronouncements on whether to use the dot operator to access properties will love the new subscripting syntax.

Software-ICs and a component marketplace

In the previous post, I was talking about Object-Oriented Programming, an Evolutionary Approach. What follows is a thought experiment based on that.

Chapter 6 of Brad Cox’s book, once he’s finished explaining how ObjC works (and who to buy it from), is concerned with his vision of how Object-Oriented software will be built. He envisions “Software-ICs”—compiled object files defining the code to support a single class (no need for header files, remember) that are distributed with documentation on how to use that class.

Developers or “software librarians” connect ICs together into collections called “categories”, which are implemented as object libraries. It’s a bit unfortunate that “category” is an inappropriate name mainly due to later reuse by NeXT; but then the alternate word “framework” is also unfortunate due to confusion with the computer science term (which allows that AppKit, WebObjects UIKit are frameworks, but Foundation, Quartz and so on are not). But it’s an entirely understandable reuse: in Smalltalk-80 related methods are grouped into categories, and NeXT used the same terminology for a very similar purpose.

Interestingly, Cox allowed for the compiler to generate vtables of selectors for each category, a bit like the Amiga operating system’s library format. That’s to support having different variable types for selectors with the same name. Modern Objective-C doesn’t support that; if you define selectors with the same name but different parameters or return values, you’ll get a warning and your code might not work correctly.

Finally, an application is a network of categories connected by the linker. One (or perhaps more, depending on your design) of the categories in the application contains the app-specific classes.

My reason for bringing this up is that this vision of object-oriented software engineering closely models component-oriented hardware engineering by allowing for software shops to produce catalogs of the components they produce at each level. Just as you can order a single IC, or a circuit board with a few ICs connected, or a whole widget, so you could order a class, or a category, or an application. If you want to build a new application, you might buy a couple of classes from one vendor, a category from another vendor, then write a few classes yourself and integrate the whole lot.

Enough ancient book, talk about the real world.

We have a lot of this, and make quite a lot of use of it. There are loads of Objective-C classes, libraries and frameworks out there for us to use, and to some extent there are catalogs. Many of the components we can use are open source, which means that we can treat the class interfaces themselves as the catalogs. If we’re lucky there’ll be some documentation, perhaps in the form of AppleDoc or a README.

Unfortunately availability vastly outstrips discoverability. You have to go to multiple catalogs to ensure that you’ve exhausted the search space: Google Code, GitHub, BitBucket, SourceForge etc. in addition to finding commercial libraries which won’t be listed in any of those places. Actual code search engines like OpenGrok and Koders are great for finding out about source code, but not so great for discovering it in the first place.

Metacatalogs like Cocoa Objects, Cocoa Controls and CocoaOpen solve part of this problem by letting people list their source code in a single place, but because they’re incomplete they only add to the number of places you need to search.

Then, once you’ve got the component, what do you do? Are you meant to drop the source files into your project? Should you drop the project in and add the library as a dependency of your app? Should you use CocoaPods?

Learn from what we already do

Just as we already push most of the apps we write to a single app store where customers can discover, purchase and install apps in a state where they’re ready to use, we should do the same with components.

[Please bear in mind that like most descriptions of ideas, a lot of nuances and complexity are known but are elided below for the sake of clarity. Comment brownie points will not be awarded for comments that explain how I haven’t considered case X; I probably have.]

A component store would, for browsers, start off very similar to Cox’s idea of a component catalog. You’d go to it and search for a component that suits your needs. You could see a “spec sheet” for each component detailing what it does, what it costs, the terms of using it and that sort of thing. You’d then buy the component if it’s paid for and download it. If the licence permits it you could download the source, too.

The download would drop the binary and headers into a folder that Xcode would recognise as an additional SDK. It would also drop the documentation in docset format into a standard location. An Xcode project would just need to point at the additional SDK and it could pick up all of the components available to the developer.

From the perspective of a component manufacturer, the component store would look a little like iTunes Connect. You’d write your code, then package it up for the store in a standard way along with the description that goes into the “spec sheet”. For open source projects that could just involve git push componentstore master to have the store itself generate the binaries and documentation from the source code.

Comparing Objective-C and Objective-C with Objective-C

A while back, I wrote an object-oriented dispatch system for Objective-C. It defines only three things: an object type (the BlockObject), a way to create new objects (the BlockConstructor), and a way to message objects (the dispatch mechanism).

That’s all that the first version of Objective-C defines, as documented in Brad Cox’s Object-Oriented Programming: an Evolutionary Approach.

Objective-C is a hybrid programming language[…]formed by grafting the Smalltalk-80 style of object-oriented programming onto a C language rootstock. Objective-C adds precisely one new data type, the object, to those C provides already, and precisely one new operation, the message expression. Like Smalltalk-80, Objective-C makes no compile-time distinction between different kinds (classes) of objects.

So if you want to try out 1980s-style OOP using Objective-C, the tool to use is not Objective-C itself but BlockObject.

That quote is from the start of Chapter Four (of the Cox edition, not the Cox and Novolbilski edition), which goes on to describe the Objective-C preprocessor, the multiple-dispatch system (using strings as message selectors like BlockObject does—and indeed like Apple’s ObjC does albeit hidden behind the opaque SEL type), and the _msg C function that implements message dispatch.

Factory Objects

Having objects is all very well, but we need some way to create them. In the BlockObject system, there’s a special type called BlockConstructor that configures and returns a new instance of an object.

That’s boring. Objective-C doesn’t define a “constructor type”, can’t we just use objects? Well, yes. Given a class definition like this:

= Stack : Object { id *objects; unsigned int count; }

Objective-C as-was automatically defines a global factory object called Stack that can be used anywhere in the code. You grab it and use it like this:

extern id Stack;

id myStack = [Stack new];
[myStack push: someObject];

From there, Objective-C won’t surprise you much. You can define instance methods:

-push:anObject {
	if (count == MAX_OBJECTS) [self cannotGrow];
	objects[count++] = anObject;
	return self;
}

and factory methods:

+new {
	id stack = [super new];
	stack->objects = malloc(MAX_OBJECTS * sizeof(id));
	return stack;
}

Though users of “modern” Objective-C will notice that there’s no compile-time type checking: no Stack *stack variable declaration for example. Indeed there’s no @interface for the class at all; all objects are of type id, you can send any message to any object and what it does with that message is up to the receiver. Just like Smalltalk or Ruby.

Building an object-oriented dispatch system in Objective-C

iTunes was messing about rebuilding the device I was trying to use for development, so I had time over lunch to write a new message dispatch system in the Objective-C language. “But wait,” you say, “Objective-C already has a message dispatch system!” True, and it’s better than the one I’ve created. But it doesn’t use blocks, and blocks are cool :-). In the discussion below, I’ll build up an implementation of a “recent items” list, which is discussed in Kevlin Henney’s presentation linked in the acknowledgements.

The constructor

One important part of an object system is the ability to make new objects. Let’s declare an object type, and a constructor type that returns one of those objects:

typedef id (^BlockObject)(NSString *selector, NSDictionary *parameters);
typedef BlockObject(^BlockConstructor)(void);

I’d better explain signature of the BlockObject type. Objects can be sent messages; what we’re doing is saying that if you execute the object with a selector name, the object will dispatch the correct implementation with the parameters you supply and will give you back the return value from the implementation. That’s what objc_msgSend() does in old-school Objective-C. The constructor is going to return this dispatch block – actually it’ll return a copy of that block, so invoking the constructor multiple times results in multiple copies of the object. Let’s see that in action.

BlockConstructor newRecentItemsList = ^ {
    BlockObject list = ^(NSString *selector, NSDictionary *parameters) {
        return (id)nil;
    }
    return (BlockObject)[list copy];
}

Yes, you have to cast nil to id. Who knew C could be so annoying?

Message dispatch

An object that can’t do anything isn’t very exciting, so we should add a way for it to look up and execute implementations. Method implementations are of the following type:

typedef id (^BlockIMP)(NSDictionary *parameters);

With that in place, I’ll show you an example of the object with a dispatch system in place, and discuss it afterward.

typedef void (^SelectorUpdater)(NSString *selector, BlockIMP implementation);

BlockConstructor newRecentItemsList = ^ {
    __block NSMutableDictionary *selectorImplementationMap = [NSMutableDictionary dictionary];
    __block SelectorUpdater setImplementation = ^(NSString *selector, BlockIMP implementation) {
        [selectorImplementationMap setObject: [implementation copy] forKey: selector];
    };

    BlockObject list = ^(NSString *selector, NSDictionary *parameters) {
        BlockIMP implementation = [selectorImplementationMap objectForKey: selector];
        return implementation(parameters);
    };
    return (BlockObject)[list copy];
};

The variables selectorImplementationMap and setImplementation are __block variables in the constructor block. This means that every time the constructor is called, the returned instance has its own copy of these variables that it is free to use and to modify. Let me put that another way: the entire message-dispatch system is encapsulated inside each instance. If a class, or even an individual instance, wants to implement message dispatch in a different way, that’s cool. It also means that an instance can change its own methods at runtime without affecting any other objects, including other instances of the same class. As long as the object still conforms to the contract that governs method dispatch, that’s cool too.

Implementing the recent items list

OK, now that we’ve got construction and messaging in place, we can start making useful objects. Here’s the implementation of the recent items list, where I’ve chosen to use an NSMutableArray for the internal storage, as with the dispatch map it’s an instance variable of the list. Needless to say you could change this to a C array, STL container or anything else without breaking external customers of the object.

BlockConstructor newRecentItemsList = ^ {
    __block NSMutableDictionary *selectorImplementationMap = [NSMutableDictionary dictionary];
    __block SelectorUpdater setImplementation = ^(NSString *selector, BlockIMP implementation) {
        [selectorImplementationMap setObject: [implementation copy] forKey: selector];
    };
    __block NSMutableArray *recentItems = [NSMutableArray array];
    setImplementation(@"isEmpty", ^(NSDictionary *parameters) {
        return [NSNumber numberWithBool: ([recentItems count] == 0)];
    });
    
    setImplementation(@"size", ^(NSDictionary *parameters) {
        return [NSNumber numberWithInteger: [recentItems count]];
    });
    
    setImplementation(@"get", ^(NSDictionary *parameters) {
        NSInteger index = [[parameters objectForKey: @"index"] integerValue];
        return [recentItems objectAtIndex: index];
    });
    
    setImplementation(@"add", ^(NSDictionary *parameters) {
        id itemToAdd = [parameters objectForKey: @"itemToAdd"];
        [recentItems removeObject: itemToAdd];
        [recentItems insertObject: itemToAdd atIndex: 0];
        
        return (id)nil;
    });
    
    BlockObject list = ^(NSString *selector, NSDictionary *parameters) {
        BlockIMP implementation = [selectorImplementationMap objectForKey: selector];
        return implementation(parameters);
    };
    return (BlockObject)[list copy];
};

Using the list

Here is an example of creating and using a recent items list. Thankfully, since writing this post literal dictionaries have appeared, so it doesn’t look so bad:

    BlockObject recentItems = newRecentItemsList();
    BOOL isEmpty = [recentItems(@"isEmpty", nil) boolValue];

    NSDictionary *getArgs = @{ @"index" : @0 };
    @try {
        id firstItem = recentItems(@"get", getArgs);
        NSLog(@"first item in empty list: %@", firstItem);
    }
    @catch (id e) {
        NSLog(@"can't get first item in empty list");
    }

Exercises for the reader

The above class is not complete. Here are some ways you could extend it, that I haven’t covered (or, for that matter, tried).

  • Implement @"isEqual". Remember that no instance can see the ivars of any other instance, so you need to use the public interface of the other object to decide whether it’s equal to this object. You’ll need to provide a new method @"respondsToSelector" in order to build @"isEqual" properly.
  • Respond to unimplemented selectors well. The implementation shown above crashes if you send an unknown message: it’ll try to dereference a NULL block. That, well, it’s bad. Objective-C objects have a mechanism that catches these messages, allowing the object a chance to lazily add a method implementation or forward the message to a different object.
  • Write an app using these objects. :-)

Credit where it’s due

This work was inspired by Kevlin Henney’s presentation: It is possible to do OOP in Java. The implementation shown here isn’t even the first time this has been done using Objective-C blocks. The Security Transforms in Mac OS X 10.7 work in a very similar way. This is probably the first attempt to badly document a bad example of the art, though.