Structure and Interpretation of Computer Programmers

I make it easier and faster for you to write high-quality software.

Friday, May 29, 2015

Dynamic Method Dispatch in Object-Oriented Programming in Functional Programming in Swift

In the previous episode, I said that objects are functions that map their ivars onto methods. However, the objects that I demonstrated in the post were tables, structures of functions that closed over the ivars, like this:

struct List<T> {
  let count: () -> Int
  let at: (Int) -> T?

Functions and tables are actually interchangeable, so this is not a problem and there’s no sleight of hand.

Here’s a simple example: imagine a function that maps Booleans (instances drawn from the set Boolean) to integers (instances drawn from the set Int):

enum Boolean {
 case False
 case True

func intValue(b : Boolean) -> Int
  switch b
  case .False:
    return 0
  case .True:
    return 1

This is, notation aside, identical to encoding the map in a table:

struct BooleanIntMap {
  let False = 0
  let True = 1

That’s the state that my objects were in at the end of the previous post. So let’s complete the story, and turn these objects from tables into functions.

Three Minute Hero

Along the way, I’ll fix a deficiency in the previous version of these objects. When you got an object, it was in the form of a table of all of its methods, but typically you only want to call one at a time. In Objective-C, you pass an object a selector and it returns the appropriate method which you then call (well, that’s how the GNU runtime works. The NeXT and Apple runtimes jump right into the returned function). See? An object really is a function that maps its variables onto the methods you want.

There are a limited (for now) number of legitimate selectors to send to an object, so they can easily be represented with an enumeration.

enum DynamicListSelectors {
  case count
  case at

Now it gets a bit complicated (Objective-C and Smalltalk are also this complicated, but use a different notation that hides it from you). My list object will be a function that maps these selectors onto methods, but the two methods have different signatures. I therefore need a type which can represent (function with the .count signature) OR (function with the .at signature).

In addition to representing enumerated constants, Swift’s enums can also sum types. Remember that a type is just a set of that type’s instances: a sum type is the union of two (or more) type sets. So the list function can return something of this type:

enum DynamicList<T> {
  case count (() -> Int)
  case at ((Int) -> T?)

There are two things to do now. One is to define the function that maps from a selector onto an instance of this sum type: this is our message dispatch function, or more succinctly, our object.

func EmptyDynamicList<T>()(_cmd: DynamicListSelectors) -> DynamicList<T>
  switch _cmd {
  case .count:
    return DynamicList<T>.count({ return 0 })
  case .at:
    return DynamicList<T>.at({ (_) in return nil})

But now we need to pull apart the enumeration in the client code to find the message implementation and invoke it. In principle, Smalltalk, Objective-C and other dynamic O-O languages have to do this too (in ObjC it works by knowing how to use function pointers of different types, and treating all the methods as function pointers), but the notation or loose type system hides it from you.

let emptyCountIMP : DynamicList<Int> = EmptyDynamicList()(_cmd: .count)
switch emptyCountIMP {
case .count(let f):
  f() // returns 0
  assertionFailure("Implementation does not match selector")

Doing that every time you call a method would get boring quickly, averted by defining a couple of convenience functions.

func countOfList<T>(list: (DynamicListSelectors) -> DynamicList<T>) -> Int
  switch list(.count) {
  case .count(let f):
    return f()
  case .at(let f):
    assertionFailure("Implementation does not match selector")
    return 0 //unreached

func elementInListAtIndex<T>(list: (DynamicListSelectors) -> DynamicList<T>, index: Int) -> T?
  switch list(.at) {
  case .at(let f):
    return f(index)
  case .count(let f):
    assertionFailure("Implementation does not match selector")
    return nil //unreached

Now it should be easy enough to use a more complex implementation of the list class. Here’s a linked list, as before the tail is itself a linked list (which is now a function mapping a selector to an implemetation).

func DynamicLinkedList<T>(head: T,
                          tail: (DynamicListSelectors) -> DynamicList<T>)
                         (_cmd: DynamicListSelectors) -> DynamicList<T>
  switch _cmd {
  case .count:
    return DynamicList<T>.count({
      return 1 + countOfList(tail)
  case .at:
    return DynamicList<T>.at({ (i) in
      if i < 0 {
        return nil
      if i == 0 {
        return head
      else {
        return elementInListAtIndex(tail, i - 1)

let unitList = DynamicLinkedList("Wow", EmptyDynamicList()) //(Function)
countOfList(unitList) // 1
elementInListAtIndex(unitList, 0) // "Wow"
elementInListAtIndex(unitList, 1) // nil

Draw the rest of the owl

Going from here to Smalltalk is a small matter of types. The object defined above is a total function of a small number of list message selectors to a small number list method implementations: every selector resolves to a method.

A Smalltalk object is also a function from selectors to methods, but from the range of all possible selectors (which are strings) to the domain of all possible function types. [Aside: as C doesn’t have closures, Objective-C’s message lookup domain is all possible function types that receive an object and selector as the first two arguments.] You probably didn’t write implementations for every possible selector, so most of these messages will be resolved to the doesNotRecognize: implementation which is where Smalltalk does its forwarding and dynamic method resolution, similar to Objective-C’s forwardInvocation: and Ruby’s method_missing.

The awesome power of the runtime. Now available in Swift.

posted by Graham at 14:57  

Friday, May 29, 2015

Object-Oriented Programming in Functional Programming in Swift

The maths behind functional programming predates computers. Once people had some experience with both of these things, they stripped them down and created object-oriented programming. It’s still possible to jettison a lot of the features of functional programming and work with the object-oriented core, and in this post I’ll do so using a subset of the Swift programming language.

Step One: Useing you’re type’s good

I’ll use this particular definition of a type: it’s a set. I could define a type completely as a set of constants:

Boolean :: {True, False}

or using constructors:

Natural :: {Zero, Successor(N : Natural)}

An element drawn from the set is an instance of the type. So True is an instance of Boolean, and Successor(Successor(Zero)) is an instance of Natural.

A particular, um, type of type is the “product” type, which represents a combination of other types. A tuple is product type: a tuple of type (Boolean, Natural) represents an instance of Boolean and an instance of Natural. The set (Boolean, Natural) or Boolean x Natural is the set of all possible combinations of Boolean and Natural.

Crucially, for the following argument, a struct is also a product type. Has been since before C was created, will be forever.

A function is a map from an input type to an output type. Really there’s more nuance in the world than that, particularly around whether the function works for all instances of the input type and whether there’s a one-to-one mapping between input and output instances, but that’s not relevant right now. All we need to know is that given an instance of the input type, applying a function to it will always result in a particular instance of the output type.

f :: InputType -> OutputType

You may think that you’ve seen functions with more than one input argument: that’s the same (roughly) as a function that takes a single tuple argument.

Step Two: Know what an object is

Defining instance variables as the values “known” inside an object, and methods as functions attached to an object that have access to its instance variables, I could say this:

An object is a collection of instance variables and methods that act on those instance variables.

But I want to put this a different way.

Given a collection of values, an object gives you a collection of methods that yield particular results based on those values.

But I want to put this a different way.

Given the input of a product type drawn from possible instance variables, an object will return a particular member of the product type drawn from possible methods.

But I want to put this a different way.

An object is a function that maps from instance variables to methods.

Step Three: Build an object

As the output of an object is a collection of methods, you can define an object entirely by its methods. In object-oriented programming, this is called “data hiding” or “encapsulation” and means that you can’t see what went into making the objects work this way, only that they do.

Starting small, think about the set data type. You only ever want to do one thing with a set: discover whether a particular element is contained in the set. That means that, for example, the type of a set of integers is equivalent to this type of function (in Swift syntax):

typealias Set = (Int) -> Bool

Now it’s possible to build trivial sets. Here’s one that contains no integers, and one that contains all the integers:

let emptySet : Set = { (_) in return false }
let universe : Set = { (_) in return true }

It’s easy to use the one method that these objects define: just call the objects as functions.

emptySet(12) //false
universe(12) //true

Step Four: Add some instance variables

I want to keep my Set method signature, because I want the objects I create to all be compatible (I really want them to all be of the same class). But I also want to be able to store and use instance variables from within the Set‘s method, so I can build some more interesting sets.

I’ll build a Set constructor, a function that returns properly-configured instances of Set. That constructor can close over the instance variables, therefore making them available from within the method. Here’s a constructor for Sets that represent contiguous ranges of Ints.

func RangeSet(lowerBound:Int, upperBound:Int)->Set
  return { (x) in (x >= lowerBound) && (x <= upperBound) }

Now it’s possible to create these Sets and use their method:

RangeSet(3, 7)(2) //false

let mySet = RangeSet(0, 2) //(Function)
mySet(1) //true
mySet(3) //false

Of course, the instance variables don’t have to be Ints. They could be anything else, including other Sets.

func UnionSet(setOne:Set, setTwo:Set) -> Set
  return { (x) in (setOne(x) || setTwo(x)) }

UnionSet(RangeSet(0, 2), RangeSet(3, 5))(4) //true

func IntersectSet(setOne:Set, setTwo:Set) -> Set
  return { (x) in (setOne(x) && setTwo(x)) }

IntersectSet(RangeSet(0, 5), RangeSet(3, 7))(5) //true

Even when a Set has another Set as an instance variable, it cannot see how that Set was constructed or what instance variables it has. It can only see the Set‘s method, because the encapsulation is working.

Step Five: Add more methods

Not all things that we might want to use as objects can be expressed as one method. A list has two methods: a count of the number of items it holds and a method to get the item at a given index. A list object therefore needs to represent the product of (count method, at method) and to let clients select which method they want to use from that product. This product could be expressed as a tuple:

typealias ListSelectors = (count:() -> Int, at:Int->AnyObject?)

Or (perhaps more usefully because the compiler for Swift allows generics in this case) as a struct:

struct List<T> {
  let count: () -> Int
  let at: (Int) -> T?

Now it’s possible to rely on the old trick of making constructor functions that return Lists:

func EmptyList<T>() -> List<T>
  return List(count: {return 0}, at: {(_) in return nil})

including the use of instance variables captured by closing over them:

func LinkedList<T>(head:T, tail:List<T>) -> List<T>
  return List(count: {return 1 + tail.count()},
    at: { index in return (index == 0) ? head : - 1)})

Step Six: Inherit from objects

The List is cool, but it’d be nice to be able to describe lists. I could create a function:

func describeList(list:List<SomeType>) -> String

but wouldn’t it be better to add description as a method on Lists? That would make the List type look like this:

struct ListWithDescription<T> {
  let count: () -> Int
  let at: (Int) -> T?
  let describe: () -> String

OK, but I don’t want to go back to all of my previous List implementations and add describe methods to them. What I really want to do is to add this method as an extension to Lists. I’ll define a type that includes a describe method and optionally implementations of the other methods too.

struct ListWithDescriptionSelectors<T> {
  let count: (() -> Int)?
  let at: ((Int) -> T?)?
  let describe: () -> String

Given a List and some ListWithDescriptionSelectors, it’s possible to build a ListWithDescription that knows how to describe itself and either inherits count and at from its List or overrides them itself.

func SubtypeListByAddingDescription<T>(prototype: List<T>,
  overrides: ListWithDescriptionSelectors<T>) ->
  let countImplementation : () -> Int
  if let count = overrides.count {
    countImplementation = count
  } else {
    countImplementation = prototype.count
  let atImplementation : (Int)->T?
  if let at = {
    atImplementation = at
  } else {
    atImplementation =
  return ListWithDescription<T>(count: countImplementation, at:     atImplementation, describe: overrides.describe)

(The slight complication with all the if lets is because the Swift compiler at time of writing wasn’t happy with using the ?? operator in their place.)

It’s now possible to put this into practice. In order to work with the elements in a List I need to know more about what type of thing they are, so here’s a specialisation of ListWithDescription that can describe a List of Strings. As ever, it has no special access to the instance variables of the List it extends and can only work with it through the published methods.

func ListOfStringsWithDescription(strings: List<String>) ->
  let describe: () -> String = {
    var output = ""
    for i in 0..<strings.count() {
      output = output.stringByAppendingString(!)
      output = output.stringByAppendingString(" ")
  return SubtypeListByAddingDescription(strings,
    ListWithDescriptionSelectors<String>(count: nil,
      at: nil,
      describe: describe))

let awesomeGreeting =
      EmptyList()))) //"World"
awesomeGreeting.describe() //"Hello, World"

Next Step: Draw some conclusions

Object-Oriented Programming is a simple, easy to use subset of Functional Programming.

Open Step: Cite references

The objects in this article (like the objects in Microsoft COM or my earlier BlockObject) are Procedural Data Types, which I discovered in Object-Oriented Programming Versus Abstract Data Types and User-defined types and procedural data structures as complementary approaches to data abstraction. The idea of objects as closures first appeared, to my knowledge, in Objects as Closures: Abstract Semantics of Object Oriented Languages. The last two of these articles were brought to my attention by my colleague Peter O’Hearn after a discussion on the topic of functional/O-O duality. I am indebted to him for his help in drawing this article out of me.

After Step: Finish this idea

This journey, as we say at Facebook, is 1% complete. There’s plenty more to explore here, and I’m not sure the explanation is as clear as it could be. It is, however, written down, which is a start.

posted by Graham at 00:39  

Tuesday, May 19, 2015

Object-Oriented Programming in Objective-C

UIKonf 1995 Keynote : Object-Oriented Programming in Objective-C


Welcome to the keynote for UIKonf 1995. I’m really excited for what 1995 will bring. Customers are upgrading to last year’s OpenStep release, which means that we get to use the new APIs and the best platform around. And really, there are no competitors. OS/2 Warp doesn’t seem to be getting any more traction than previous versions, and indeed Microsoft seems to be competing against its own products with Windows. Their biggest release this year, the delayed Windows 93, looks like being a warmed-over version of MS-DOS and Windows 3, which certainly can’t be compared with a full Unix system like OpenStep. 1995 will go down in history as the year of NeXT on the desktop.

I want to talk about a crisis in software design, and that is the object-oriented crisis. Well, I suppose it isn’t really, it’s the procedural crisis again. But now we pretend that our procedural code is object-oriented, and Objective-C is the weapon that enacts this travesty.

What’s the main benefit of Objective-C over Smalltalk? It’s C. Rather than attempt to graft some foreign function interface onto Smalltalk and make us write adaptors for all of our C code, Brad Cox had the insight that he could write a simple message-sending library and a syntax preprocessor on the C language, and let us add all of the object-oriented programming we’d get from Smalltalk on top of our existing C code.

What’s the main drawback of Objective-C over Smalltalk? It’s C. Rather than being able to rely on the object-oriented properties of programs to help us understand them, we can just write a load of C code that we wrap up in methods and call it “object-oriented”. This is the source of our crisis. In 1992, Brad Cox claimed that Object-Oriented Programming (or Message/Object Programming as he also called it) was “the silver bullet” that Fred Brooks claimed didn’t exist. That it would help us componentise software into isolated units that can be plugged together, like integrated circuits bought from a catalogue and assembled into a useful product on a circuit board.

This idea of encapsulated software components is not new, and was presented at the NATO conference on software engineering in 1968. At that conference, Doug McIlroy presented an invited paper called “Mass-Produced Software Components” in which he suggested that the software industry would not be “industrialised” until it supported a subindustry of comprehensible, interchangeable, well-specified components. Here’s the relevant quote, the gendered pronouns are from the original.

The most important characteristic of a software components industry is that it will offer families of routines for any given job. No user of a particular member of a family should pay a penalty, in unwanted generality, for the fact that he is employing a standard model routine. In other words, the purchaser of a component from a family will choose one tailored to his exact needs. He will consult a catalogue offering routines in varying degrees of precision, robust- ness, time-space performance, and generality. He will be confident that each routine in the family is of high quality — reliable and efficient. He will expect the routine to be intelligible, doubtless expressed in a higher level language appropriate to the purpose of the component, though not necessarily instantly compilable in any processor he has for his machine. He will expect families of routines to be constructed on rational principles so that families fit together as building blocks. In short, he should be able safely to regard components as black boxes.

Is Object-Oriented Programming actually that silver bullet? Does it give us the black-box component catalogue McIlroy hoped for? Most of us will never know, because most of us aren’t writing Object-Oriented software.

I think this gulf between Object-Oriented design and principles as described during its first wave by people like Alan Kay, Brad Cox, and Bertrand Meyer is only going to broaden, as we dilute the ideas so that we can carry on programming in C. When I heard that a team inside Sun had hired a load of NeXT programmers and were working on a post-Objective-C OO programming environment, I was excited. We know about the problems in Objective-C, the difficulties with working with both objects and primitive types, and the complexity of allowing arbitrary procedural code in methods. When the beta of Oak came out this year I rushed to request a copy to try.

Java, as they’ve now renamed it, is just an easier Objective-C. It has garbage collection, which is very welcome, but otherwise is no more OO than our existing tools. You can still write C functions (called static methods). You can still write long, imperative procedures. There are still primitive types, and Java has a different set of problems associated with matching those up to its object types. Crucially, Java makes it a lot harder to do any higher-order messaging, which means that if it becomes adopted we’ll see a lot more C and a lot less OO.

The meat

I thought I’d write an object-oriented program in modern, OPENSTEP Objective-C, just to see whether it can even be done. I won’t show you all the code, but I will show you the highlights where the application looks strongly OO, and the depths where it looks a lot like C. I don’t imagine that you’ll instantly go away and rewrite all of your code, indeed many of you won’t like my style because it’s so different from usual Objective-C. My hope is that it gives you a taste of what OOP can be about, inspires you to reflect on your own approach, and encourages you to take more from other ideas about programming than some sugar that superficially wraps C.

Despite a suggestion from Marvin, the paranoid android, I’m going to talk to you about Life.


Always be returning

Every method in this application returns a value, preferably an object, except where that isn’t allowed due to assumptions made by the OpenStep frameworks. Where the returned object is of the same type, the client code should use the returned object, even if the implementation is to return self. Even things that look like mutators follow this pattern: with the effect that it doesn’t matter to a client whether an object is mutable or immutable because it’ll use it the same way. Compare a mutable object:

  foo = aFoo;
  return self;

with an immutable one:

  return [self copyWithReplacementFoo:aFoo];

and the client doesn’t know which it’s using:

[[myThing setFoo:aDifferentFoo] doSomeStuff];

That means it’s easy to change the implementation between these two choices, without affecting the client code (which doesn’t need to assume it can keep messaging the same object). In Life, I built a mutable Grid class, but observed that it could be swapped for an immutable one. I didn’t need to change the app or the tests to make it happen.

This obviously has its limitations, but within those limitations is a great approach. If multiple objects in a system share a collaborator, then you need to choose whether that collaborator is mutable (in which case all correlated objects see all updates, even those made elsewhere in the system) or immutable (in which case collaborating objects will only see their own updates, unless some extra synchronisation is in place). Using the same interface for both lets you defer this decision, or even try both to understand the implications.

Always be messaging

In this Life application there are no C control statements: no if, no for, no while. Everything is done by sending messages to objects. Two examples are relevant, and both come from the main logic of the “game”. If you’ve seen the rules of Life, you’ve probably seen them expressed in a form like this:

If a cell is dead and has X neighbours, it becomes living otherwise it remains dead. If a cell is living and has Y neighbours, it remains living otherwise it becomes dead.

It seems that there are three if statements to be written: one to find out whether a cell is living or dead, and another in each branch to decide what the next state should be.

The first can be dealt with using the class hierarchy. A class (at some theoretical level what I mean here is “a constructor”, but that term is not really used in Objective-C so I’m going to say “class” which is the closest term) can often be a substitute for an if statement, by replacing the condition with polymorphism.

In this case, the question “is a cell living or dead” can be answered by sending a message to either a DeadCell or a LivingCell. The question disappears, because it was sent to an object that pre-emptively knew the answer.

@interface LivingCell : Cell

@interface DeadCell : Cell

Now each can answer the question “what is my next state?” without needing to test what its current state is. But how to do that without that other if statement? There’s a finite number of possible outcomes, keyed on an integer (the number of living neighbours the cell has), which means it’s a question that can be answered by sending a message to an array. Here’s how the Cell does it.

-tickOnGrid:grid atX:(NSInteger)x y:(NSInteger)y;
  return [[self potentialStates]
         [self neighboursOnGrid:grid atX:x y:y]];

Each of the two subclasses of Cell knows what its potential states are:

static id nextStatesFromLiving;

  nextStatesFromLiving = [[NSArray alloc] initWithObjects:

-potentialStates { return nextStatesFromLiving; }

Why write the program like this? To make it easier to understand. There’s an idea in the procedural programming world that the cyclomatic complexity – the number of conditions and loops in a function – should be minimised. If all a method is doing is messaging other objects, the complexity is minimal: there are no conditional paths, and no loops.

My second example is how a loop gets removed from a method: with recursion, and appropriate choice of target object. One of the big bones of contention early on in NeXT’s use of Objective-C was changing the behaviour of nil. In Smalltalk, this happens:

nil open.
MessageNotUnderstood: receiver of "open" is nil

In Objective-C, nothing happens (by default, anyway). But this turns out to be useful. Just as the class of an object can stand in for conditional statements, the nil-ness of an object can stand in for loop termination. A method does its thing, and sends itself to the receiver of the next iteration. When that receiver becomes nil, the message is swallowed and the loop stops.

All loops in Life that are based on doing something a certain number of times are based on a category method on NSNumber.

@implementation NSNumber (times)

-times:target perform:(SEL)action
  return [self times:target perform:action withObject:nil];

-times:target perform:(SEL)action withObject:object
  return [[self nonZero] realTimes:target

-realTimes:target perform:(SEL)action withObject:object
  [target performSelector:action withObject:object];
  return [[NSNumber numberWithInteger:[self integerValue] - 1]
       times:target perform:action withObject:object];

  return ([self integerValue] != 0)?self:nil;


Notice that the conditional expression doesn’t violate the “no if statements” rule, because it’s an expression not a statement. There’s one thing that happens, that yields one of two values. The academic functional programming community recently rallied around the Haskell language, which provides the same distinction: no conditional statements, easy conditional expressions.

Never be sequencing

Related to the idea of keeping methods straightforward to understand is ensuring that they don’t grow too long. Ideally a method is one line long: it returns the result of messaging another object. Then there are a couple of slightly larger cases:

  • a fluent paragraph, in which the result of one message is the receiver for another message. If this gets too deep, though, your method probably has too many coupled concerns associated with a Law of Demeter violation.
  • something that would be a fluent paragraph, but you’ve introduced a local variable to make it easier to read (or debug; Objective-C debuggers are not good with nested messages).
  • a method sends a message, then returns another more relevant object. Common examples replace some collection object then return self.

Finally, there are times when you just can’t do this because you depend on some API that won’t let you. A common case when working with AppKit is methods that return void; you’re forced to sequence them because you can’t compose them. The worst offender in Life is the drawing code which uses AppKit’s supposedly object-oriented API, but which is really just a sequence of procedures wrapped in message-sending sugar (that internally use some hidden context that was set up on the way into the drawRect: method).

  float beginningHorizontal = (float)x/(float)n;
  float beginningVertical = (float)y/(float)n;
  float horizontalExtent = 1.0/(float)n;
  float verticalExtent = 1.0/(float)n;
  NSSize boundsSize = [view bounds].size;
  NSRect cellRectangle = 
           beginningHorizontal * boundsSize.width,
           beginningVertical * boundsSize.height,
           horizontalExtent * boundsSize.width,
           verticalExtent * boundsSize.height
  NSBezierPath *path =  [NSBezierPath bezierPathWithRect:cellRectangle];
  [[NSColor colorWithCalibratedWhite:[denizen population] alpha:1.0] set];
  [path stroke];
  [[NSColor colorWithCalibratedWhite:(1.0 - [denizen population])   alpha:1.0] set];
  [path fill];

The underlying principle here is “don’t make a load of different assignments in one place”, whether those are explicit with the = operator, or implicitly hidden behind message-sends. And definitely, as suggested by Bertrand Meyer, don’t combine assignment with answering questions.

Higher-order messages

You’ve already seen that Life’s loop implementation uses the target-action pattern common to AppKit views, and so do the views in the app. This is a great way to write an algorithm once, but leave it open to configuration for use in new situations.

It’s also a useful tool for reducing boilerplate code and breaking up complex conditional statements: if you can’t represent each condition by a different object, represent them each by a different selector. An example of that is the menu item validation in Life, which is all implemented on the App delegate.

  id action = NSStringFromSelector([menuItem action]);
  SEL validateSelector = NSSelectorFromString([@"validate"
  return [[self performSelector:validateSelector withObject:menuItem]

  return @(timer == nil);

  return @(timer != nil);

  return @(timer == nil);

  return @(YES);

We have four simple methods that document what menu item they’re validating, and one pretty simple method (a literate paragraph that’s been expanded with local variables) for choosing one of those to run.

There’s no message-forwarding needed in the Life app, and indeed you can do some good higher-order messaging without ever needing to use it. You could imagine a catch-all for unhandled -validate<menuitem>: messages in the above code, which might be useful.

Objective-C’s type checks are wrong for Objective-C

Notice that all of the examples I’ve shown here have used the NeXTSTEP <=3.3 style of method declaration, with (implicit) id return and parameter types. That’s not because I’m opposed to type systems, although I am opposed to using one that introduces problems instead of solving them.

What do I want to do with objects? I want to send messages to receivers. Therefore, if I have a method that sends a message to an object, what I care about is whether the recipient can handle that message (I don’t even care whether the handling is organised upfront, or at runtime). The OPENSTEP compiler’s type checks let me know whether an object is of a particular class, or conforms to a particular protocol, but both of these are more restrictive than the tests that I actually need. Putting the types in leads to warnings in correct code more than it leads to detection of incorrect code, so I leave them out.

Of course, as with Objective-C’s successes, Objective-C’s mistakes also have predecessors in the Smalltalk world. As of last year (1994) the Self language used runtime type-feedback to inline message-sending operations (something that Objective-C isn’t doing…yet, though I note that in Java all “message sends” are inline function calls instead) and there’s an ongoing project to rewrite the “Blue Book” library to allow the same in a Smalltalk environment.

What is really needed is a form of “structural type system”, where the shape of a parameter is considered rather than the name of the parameter. I don’t know of any currently-available object-oriented languages that have something similar to this, but I’ve heard INRIA are working on an OO variant of Caml that has structural typing.

But what about performance?

It’s important to separate object-oriented design, which is what I’ve advocated here, from object-oriented implementation, which is the code I ended up with as a result. What the design gave me was:

  • freedom to change implementations of various algorithms (there are about twice as many commits to production code in Life as there are to test code)
  • hyper-cohesive and decoupled objects or categories
  • very short and easy to understand methods

The implementation ends up in a particular corner of the phase space:

  • greater stack depths
  • lots of method resolutions
  • lots of objects
  • lots of object replacements

but if any of this becomes a problem, it should be uninvasive to replace an object or a cluster of objects with a different implementation because of the design choices made. This is where Objective-C’s C becomes useful: you can rip out all of that weird nested message stuff and just write a for loop if that’s better. Just because you shouldn’t start there, doesn’t mean you can’t end up there.

An example of this in Life was that pretty early in development, I realised the Cell class didn’t actually need any state. A Grid could tell a Cell what the surrounding population was, and the Cell was just the algorithm that encapsulated the game’s rules. That meant I could turn a Grid of individual Cells into a Flyweight pattern, so there are only two instances of Cell in the whole app.


I’m not so much worried for OOP itself, which is just one way of thinking about programs without turning them into imperative procedures. I’m worried for our ability to continue writing quality software.

Compare my list of OO properties with a list of functional programming properties, and you’ll see the same features: immutable values, composed functions, higher-order functions, effectless reasoning over effectful sequencing. The crisis of 1995 shows that we took our existing procedural code, observed these benefits of OOP, then carried on writing procedural code while calling it OOP. That, unsurprisingly, isn’t working, and I wouldn’t be surprised if people blame it on OOP with its mutable values, side-effecting functions, and imperative code.

But imagine what’ll happen when we’ve all had enough of OOP. We’ll look to the benefits some other paradigm, maybe those of functional programming (which are, after all, the same as the benefits of OOP), and declare how it’s the silver bullet just as Brad Cox did. Then we’ll add procedural features to functional programming languages, or the other way around, to reduce the “learning curve” and make the new paradigm accessible. Then we’ll all carry on writing C programs, calling them the new paradigm. Then that won’t work, and we’ll declare functional programming (or whatever comes next) as much of a failure as object-oriented programming. As we will with whatever comes after that.

Remember, there is no silver bullet. There is just complexity to be governed.

posted by Graham at 17:30  

Tuesday, May 12, 2015

Programming is not a craft

I agree with this, programming is not a craft by Dan North.

So here’s my concern with the idea of Software Craftsmanship. It’s at risk of letting programmers’ egos run riot. And when that happens… well, the last time they went really nuts we got Web Services, before that J2EE.


The best software should be understated and unobtrusive (as, maybe, should be the best programmers).

posted by Graham at 17:57  

Friday, May 8, 2015

Did that work? Maybe.

A limitation with yesterday’s error-preserving approach is that it leaves you on your own to recover from problems. Assuming your error definitions are sufficiently granular, this should be straightforward but tedious. Find out what went wrong, recover from it, then replay everything that happened afterwards.

Recovering from failures automatically is difficult in general, after all, if we could do that there wouldn’t have been a failure in the first place. But there’s no reason to then have a bunch of code that replays the rest of the workflow from the point of recovery, you’ve already written that code on the happy path.

Why can’t we just do that? On an error, why don’t we recover from the immediate error then go back in time to the point where it occurred and start again with our recovered value? Seems straightforward enough, let’s do it. In doing so, let’s borrow (at least in a superficial fashion) the idea of the optional object.

A Maybe object represents the possibility that a value exists, and the ability to find out whether it does. Should it not contain a value, you should be able to find out why not.

@interface Maybe : NSObject

@property (nonatomic, strong, readonly) NSError *error;

- (BOOL)hasValue;
- recoverWithStartingValue:value;

+ just:value;
+ none:aClass error:(NSError *)anError;


I have a value

OK, let’s say that what you wanted to do succeeded, and you want to use the result object. It’d be kindof sucky if you had to test whether the Maybe contained a value at every step of a long operation, and unwrap it to get the object out that you care about. So let’s not make you do that.

@interface Just : Maybe

@implementation Just
    id _value;

-initWithValue:value {
    self = [super init];
    if (self) {
        _value = value;
    return self;

-(NSError *)error {
    NSAssert(NO, @"No error in success case");
    return nil;
-(BOOL)hasValue { return YES; }
-recoverWithStartingValue:value {
    NSAssert(NO, @"Cannot recover from success");
    return nil;
-forwardingTargetForSelector:(SEL)aSelector { return _value; }


OK, if everything succeeds you can use the Maybe result (which will be Just the value) as if it is the value itself.

I don’t have a value

The other case is that your operation failed, so we need to represent that. We need to know what type of object you don’t have(!), which will be useful because we can then treat the lack of value as if it is an instance of the value. How will this work? In None, the no-value version of a Maybe, we’ll just absorb every message you send.

@interface None : Maybe
-initWithClass:aClass error:(NSError *)anError;

@implementation None
    Class _class;
    NSError *_error;
    NSMutableArray *_invocations;

-initWithClass:aClass error:(NSError *)anError {
    self = [super init];
    if (self) {
        _class = aClass;
        _error = anError;
        _invocations = [NSMutableArray array];
    return self;

-(NSError *)error { return _error; }
-(BOOL)hasValue { return NO; }

-methodSignatureForSelector:(SEL)aSelector {
    return [_class instanceMethodSignatureForSelector:aSelector];

-(void)forwardInvocation:(NSInvocation *)anInvocation {
    id returnValue = self;
    [_invocations addObject:anInvocation];
    [anInvocation setReturnValue:&returnValue];

-recoverWithStartingValue:value {
    id nextObject = value;
    while([_invocations count]) {
        id invocation = [_invocations firstObject];
        [_invocations removeObjectAtIndex:0];
        [invocation invokeWithTarget:nextObject];
        [invocation getReturnValue:&nextObject];
    return nextObject;


Again, there’s no need to unwrap a None (and that makes no sense, because it doesn’t contain anything). You just use it as if it is the thing that it represents (a lack of).


You go through your complex process, and somewhere along the way it failed. At this point your Maybe answer turned into None, because you didn’t get a value at some step. But it carried on paying attention to what you wanted to do.

Now it’s time to turn back the clock. Looking at the error I get from the None, I can see what step failed and what I need to do to be in a position to try again. When I make that happen, I’ll get some object which would’ve been valid at that intermediate point in my operation.

Because the None was paying attention to what I tried to do to it, I can replay the whole process from the point where it failed, using the result from my recovery path.

Worked Example

Here’s an object that requires two steps to use. Either step could fail, and one of them does unless you take action to fix it up.

@interface AnObject : NSObject

@property (nonatomic, assign, getter=isRecovered) BOOL recovered;



@implementation AnObject

    return [self isRecovered] ? [Maybe just:self] :
        [Maybe none:[self class] error:[NSError errorWithDomain:@"Nope"

    return [Maybe just:@"Winning"];


In using this object, I want to compose the two steps. If it goes wrong, I know what to do to recover, but I don’t want to have to explicitly write out the workflow twice if I got it right the first time.

int main(int argc, char *argv[]) {
    @autoreleasepool {
        id anObject = [AnObject new];
        id result = [[anObject this] that];
        if ([result hasValue]) {
            NSLog(@"success: %@", [result lowercaseString]);
        } else {
            NSLog(@"failed with error %@, recovering...", [result error]);
            [anObject setRecovered:YES];
            result = [result recoverWithStartingValue:anObject];
            NSLog(@"ended up with %@", [result uppercaseString]);


The NSError-star-star convention lets you compose possibly-failing messages and find out either that it succeeded, or where it went wrong. But it doesn’t encapsulate what would have happened had it gone right, so you can’t just rewind time to where things failed and try again. It is possible to do so, simply by encapsulating the idea that something might work…maybe.

posted by Graham at 21:03  

Thursday, May 7, 2015

Getting better at doing it wrong

For around a month at the end of last year, I kept a long text note called “doing doing it wrong right”. I was trying to understand error handling in programming, look at some common designs and work out a plan for cleaning up some error-handling code I was working with myself (mercifully someone else, with less analysis paralysis, has taken on that task now).

Deliciously the canonical writing in this field is by an author with the completely apt name Goodenough. His Structured Exception Handling and Exception Handling: Issues and a Proposed Notation describe the problem pretty completely and introduce the idea of exceptions that can be thrown on an interesting condition and caught at the appropriate level of abstraction in the caller.

As an aside, his articles show that exception handling can be used for general control flow. Your long-running download task can throw the “I’m 5% complete now” exception, which is caught to update the UI before asking the download to continue. Programming taste moved away from doing that.

In the Cocoa world, exceptions have never been in favour, probably because they’re too succinct. In their place, multi-if statement complex handling code is introduced:

NSError *error = nil;
id thing = [anObject giveMeAThing:&error];
if (!thing) {
  [self handleError:error];
id otherThing = [thing doYourThing:&error];
if (!otherThing) {
  [self handleError:error];
id anotherThing = [otherThing someSortOfThing:&error];

…and so it goes.

Yesterday in his NSMeetup talk on Swift, Saul Mora reminded me of the nil sink pattern in Objective-C. Removing all the error-handling from the above, a fluent (give or take the names) interface would look like this:

id anotherThing = [[[anObject giveMeAThing] doYourThing] someSortOfThing];

The first method in that chain to fail would return nil, which due to the message-sink behaviour means that everything subsequent to it preserves the nil and that’s what you get out. Saul had built an equivalent thing with option types, and a function Maybe a -> (a -> Maybe b) -> Maybe b to hide all of the option-unwrapping conditionals.

Remembering this pattern, I think it’s possible to go back and tidy up my error cases:

NSError *error = nil;
id anotherThing = [[[anObject giveMeAThing:&error]
if (!anotherThing) {
  [self handleError:error];

Done. Whichever method goes wrong sets the error and returns nil. Everything after that is sunk, which crucially means that it can’t affect the error. As long as the errors generated are specific enough to indicate what went wrong, whether it’s possible to recover (and if so, how) and whether anything needs cleaning up (and if so, what) then this approach is…good enough.

posted by Graham at 16:11  

Friday, May 1, 2015


It’s been almost a year since my first day at Facebook, sitting in an overcrowded meeting room with my bootcamp class because 42 Earlham Street was full and it’d be another week before we moved to 10 Brock Street, with its gargantuan empty spaces (which are no longer empty: nearly half the company has joined since I started).

How has that year been, you ask? Hard. It’s been hard. Fun, worthwhile, educational, exciting, but hard. Facebook is very different from any company I’ve worked at before: it’s bigger, it’s faster, it’s more ambitious. Everything I had learned about making software outside the company was…well no, not wrong, certainly not wrong. But it was perhaps inapplicable. Facebook does things differently, I can’t immediately have impact by doing what I did at my other employers, how can I cope with this?

When you’re used to the life of minor engineering celebrity, where people go to conferences because you’re on the bill, and your pithy statements on programming get retweeted up the wazoo, it’s easy to get hubristic. I did: I know what I’m doing, the engineers here don’t yet know that, and there will be a glorious revelation when I bring the two tablets (an iPad and a Galaxy Note 10, I don’t know) down from the mountain, sell everyone in the company a copy of my book and we all start doing things The Right Way™.

That wasn’t going to work. Not because what I wanted to do couldn’t work, but because I was starting from the wrong place. How many billion-dollar companies had I written software for when I started at Facebook? None. So who was I to say that my way of writing software was “right”? All I could say was that it definitely felt right.

What I had to learn to move past this was that Facebook works on data. If I can show that there’s time being lost, or bugs being introduced, or run times being lengthened, and that what I want to do would save time, or catch bugs, or speed things up, then Facebook will listen.

The opposite of “my way is not incontrovertibly correct” is not “my way is incontrovertibly incorrect”, but it was easy to come to that conclusion too, and to rage-quit the idea of having any effect here. I think I avoided this, but I also think I came pretty close to it. I could easily have decided that while I don’t like Facebook’s way, I need to suck it up and work that way. That’s wrong for the same reason that trying to boil Facebook’s oceans was wrong: just as what I’m used to doing isn’t necessarily correct, what Facebook is doing isn’t necessarily correct. There’s room for change.

Indeed, Facebook engineering is far from change-averse. Everything is set up to let you make changes, from the extreme collective code ownership (“bored changing the code you normally work on? Change the rest of it!”) to incident review (“OK, that change wasn’t the best, let’s see what we can learn from it”).

So, it took me a long time, perhaps longer than it should: I used my first year to realise that I can change Facebook and Facebook can change me, and the way we’ll make this happen is by showing each other how the changes will make us better. Now, as David Bowie said, it’s time to turn and face the strange.

posted by Graham at 19:31  

Powered by WordPress