I agree with this, programming is not a craft by Dan North.

So here’s my concern with the idea of Software Craftsmanship. It’s at risk of letting programmers’ egos run riot. And when that happens… well, the last time they went really nuts we got Web Services, before that J2EE.

[…]

The best software should be understated and unobtrusive (as, maybe, should be the best programmers).

Posted on by Graham | Leave a comment

Did that work? Maybe.

A limitation with yesterday’s error-preserving approach is that it leaves you on your own to recover from problems. Assuming your error definitions are sufficiently granular, this should be straightforward but tedious. Find out what went wrong, recover from it, then replay everything that happened afterwards.

Recovering from failures automatically is difficult in general, after all, if we could do that there wouldn’t have been a failure in the first place. But there’s no reason to then have a bunch of code that replays the rest of the workflow from the point of recovery, you’ve already written that code on the happy path.

Why can’t we just do that? On an error, why don’t we recover from the immediate error then go back in time to the point where it occurred and start again with our recovered value? Seems straightforward enough, let’s do it. In doing so, let’s borrow (at least in a superficial fashion) the idea of the optional object.

A Maybe object represents the possibility that a value exists, and the ability to find out whether it does. Should it not contain a value, you should be able to find out why not.

@interface Maybe : NSObject

@property (nonatomic, strong, readonly) NSError *error;

- (BOOL)hasValue;
- recoverWithStartingValue:value;

+ just:value;
+ none:aClass error:(NSError *)anError;

@end

I have a value

OK, let’s say that what you wanted to do succeeded, and you want to use the result object. It’d be kindof sucky if you had to test whether the Maybe contained a value at every step of a long operation, and unwrap it to get the object out that you care about. So let’s not make you do that.

@interface Just : Maybe
-initWithValue:value;
@end

@implementation Just
{
    id _value;
}

-initWithValue:value {
    self = [super init];
    if (self) {
        _value = value;
    }
    return self;
}

-(NSError *)error {
    NSAssert(NO, @"No error in success case");
    return nil;
}
-(BOOL)hasValue { return YES; }
-recoverWithStartingValue:value {
    NSAssert(NO, @"Cannot recover from success");
    return nil;
}
-forwardingTargetForSelector:(SEL)aSelector { return _value; }

@end

OK, if everything succeeds you can use the Maybe result (which will be Just the value) as if it is the value itself.

I don’t have a value

The other case is that your operation failed, so we need to represent that. We need to know what type of object you don’t have(!), which will be useful because we can then treat the lack of value as if it is an instance of the value. How will this work? In None, the no-value version of a Maybe, we’ll just absorb every message you send.

@interface None : Maybe
-initWithClass:aClass error:(NSError *)anError;
@end

@implementation None
{
    Class _class;
    NSError *_error;
    NSMutableArray *_invocations;
}

-initWithClass:aClass error:(NSError *)anError {
    self = [super init];
    if (self) {
        _class = aClass;
        _error = anError;
        _invocations = [NSMutableArray array];
    }
    return self;
}

-(NSError *)error { return _error; }
-(BOOL)hasValue { return NO; }

-methodSignatureForSelector:(SEL)aSelector {
    return [_class instanceMethodSignatureForSelector:aSelector];
}

-(void)forwardInvocation:(NSInvocation *)anInvocation {
    id returnValue = self;
    [_invocations addObject:anInvocation];
    [anInvocation setReturnValue:&returnValue];
}

-recoverWithStartingValue:value {
    id nextObject = value;
    while([_invocations count]) {
        id invocation = [_invocations firstObject];
        [_invocations removeObjectAtIndex:0];
        [invocation invokeWithTarget:nextObject];
        [invocation getReturnValue:&nextObject];
    }
    return nextObject;
}

@end

Again, there’s no need to unwrap a None (and that makes no sense, because it doesn’t contain anything). You just use it as if it is the thing that it represents (a lack of).

Recovering

You go through your complex process, and somewhere along the way it failed. At this point your Maybe answer turned into None, because you didn’t get a value at some step. But it carried on paying attention to what you wanted to do.

Now it’s time to turn back the clock. Looking at the error I get from the None, I can see what step failed and what I need to do to be in a position to try again. When I make that happen, I’ll get some object which would’ve been valid at that intermediate point in my operation.

Because the None was paying attention to what I tried to do to it, I can replay the whole process from the point where it failed, using the result from my recovery path.

Worked Example

Here’s an object that requires two steps to use. Either step could fail, and one of them does unless you take action to fix it up.

@interface AnObject : NSObject

@property (nonatomic, assign, getter=isRecovered) BOOL recovered;

-this;
-that;

@end

@implementation AnObject

-this
{
    return [self isRecovered] ? [Maybe just:self] :
        [Maybe none:[self class] error:[NSError errorWithDomain:@"Nope"
                                                           code:23
                                                       userInfo:@{}]];
}

-that
{
    return [Maybe just:@"Winning"];
}

@end

In using this object, I want to compose the two steps. If it goes wrong, I know what to do to recover, but I don’t want to have to explicitly write out the workflow twice if I got it right the first time.

int main(int argc, char *argv[]) {
    @autoreleasepool {
        id anObject = [AnObject new];
        id result = [[anObject this] that];
        if ([result hasValue]) {
            NSLog(@"success: %@", [result lowercaseString]);
        } else {
            NSLog(@"failed with error %@, recovering...", [result error]);
            [anObject setRecovered:YES];
            result = [result recoverWithStartingValue:anObject];
            NSLog(@"ended up with %@", [result uppercaseString]);
        }
    }
}

Conclusion

The NSError-star-star convention lets you compose possibly-failing messages and find out either that it succeeded, or where it went wrong. But it doesn’t encapsulate what would have happened had it gone right, so you can’t just rewind time to where things failed and try again. It is possible to do so, simply by encapsulating the idea that something might work…maybe.

Posted in code-level, OOP | 4 Comments

Getting better at doing it wrong

For around a month at the end of last year, I kept a long text note called “doing doing it wrong right”. I was trying to understand error handling in programming, look at some common designs and work out a plan for cleaning up some error-handling code I was working with myself (mercifully someone else, with less analysis paralysis, has taken on that task now).

Deliciously the canonical writing in this field is by an author with the completely apt name Goodenough. His Structured Exception Handling and Exception Handling: Issues and a Proposed Notation describe the problem pretty completely and introduce the idea of exceptions that can be thrown on an interesting condition and caught at the appropriate level of abstraction in the caller.

As an aside, his articles show that exception handling can be used for general control flow. Your long-running download task can throw the “I’m 5% complete now” exception, which is caught to update the UI before asking the download to continue. Programming taste moved away from doing that.

In the Cocoa world, exceptions have never been in favour, probably because they’re too succinct. In their place, multi-if statement complex handling code is introduced:

NSError *error = nil;
id thing = [anObject giveMeAThing:&error];
if (!thing) {
  [self handleError:error];
  return;
}
id otherThing = [thing doYourThing:&error];
if (!otherThing) {
  [self handleError:error];
  return;
}
id anotherThing = [otherThing someSortOfThing:&error];

…and so it goes.

Yesterday in his NSMeetup talk on Swift, Saul Mora reminded me of the nil sink pattern in Objective-C. Removing all the error-handling from the above, a fluent (give or take the names) interface would look like this:

id anotherThing = [[[anObject giveMeAThing] doYourThing] someSortOfThing];

The first method in that chain to fail would return nil, which due to the message-sink behaviour means that everything subsequent to it preserves the nil and that’s what you get out. Saul had built an equivalent thing with option types, and a function Maybe a -> (a -> Maybe b) -> Maybe b to hide all of the option-unwrapping conditionals.

Remembering this pattern, I think it’s possible to go back and tidy up my error cases:

NSError *error = nil;
id anotherThing = [[[anObject giveMeAThing:&error]
                               doYourThing:&error]
                           someSortOfThing:&error];
if (!anotherThing) {
  [self handleError:error];
}

Done. Whichever method goes wrong sets the error and returns nil. Everything after that is sunk, which crucially means that it can’t affect the error. As long as the errors generated are specific enough to indicate what went wrong, whether it’s possible to recover (and if so, how) and whether anything needs cleaning up (and if so, what) then this approach is…good enough.

Posted in architecture of sorts, code-level, OOP | 2 Comments

Ch-ch-ch-ch-changes

It’s been almost a year since my first day at Facebook, sitting in an overcrowded meeting room with my bootcamp class because 42 Earlham Street was full and it’d be another week before we moved to 10 Brock Street, with its gargantuan empty spaces (which are no longer empty: nearly half the company has joined since I started).

How has that year been, you ask? Hard. It’s been hard. Fun, worthwhile, educational, exciting, but hard. Facebook is very different from any company I’ve worked at before: it’s bigger, it’s faster, it’s more ambitious. Everything I had learned about making software outside the company was…well no, not wrong, certainly not wrong. But it was perhaps inapplicable. Facebook does things differently, I can’t immediately have impact by doing what I did at my other employers, how can I cope with this?

When you’re used to the life of minor engineering celebrity, where people go to conferences because you’re on the bill, and your pithy statements on programming get retweeted up the wazoo, it’s easy to get hubristic. I did: I know what I’m doing, the engineers here don’t yet know that, and there will be a glorious revelation when I bring the two tablets (an iPad and a Galaxy Note 10, I don’t know) down from the mountain, sell everyone in the company a copy of my book and we all start doing things The Right Way™.

That wasn’t going to work. Not because what I wanted to do couldn’t work, but because I was starting from the wrong place. How many billion-dollar companies had I written software for when I started at Facebook? None. So who was I to say that my way of writing software was “right”? All I could say was that it definitely felt right.

What I had to learn to move past this was that Facebook works on data. If I can show that there’s time being lost, or bugs being introduced, or run times being lengthened, and that what I want to do would save time, or catch bugs, or speed things up, then Facebook will listen.

The opposite of “my way is not incontrovertibly correct” is not “my way is incontrovertibly incorrect”, but it was easy to come to that conclusion too, and to rage-quit the idea of having any effect here. I think I avoided this, but I also think I came pretty close to it. I could easily have decided that while I don’t like Facebook’s way, I need to suck it up and work that way. That’s wrong for the same reason that trying to boil Facebook’s oceans was wrong: just as what I’m used to doing isn’t necessarily correct, what Facebook is doing isn’t necessarily correct. There’s room for change.

Indeed, Facebook engineering is far from change-averse. Everything is set up to let you make changes, from the extreme collective code ownership (“bored changing the code you normally work on? Change the rest of it!”) to incident review (“OK, that change wasn’t the best, let’s see what we can learn from it”).

So, it took me a long time, perhaps longer than it should: I used my first year to realise that I can change Facebook and Facebook can change me, and the way we’ll make this happen is by showing each other how the changes will make us better. Now, as David Bowie said, it’s time to turn and face the strange.

Posted in Uncategorized | Leave a comment

A kata above

The code kata is a method software craftspeople use to practice their craft. The idea is that you take a problem you understand, like FizzBuzz or Conway’s Life, and build an application that implements it. Then build another one. And another. Pretty soon, you should be quite good at building a thing with well-understood requirements.

But that’s the problem, isn’t it? Our professional time can be split into two parts:

  • the part where we don’t think we quite understand what we’re meant to be doing.
  • the part where we are wrong.

So what’s the point of being good at implementing well-understood requirements effortlessly? A partial answer is that being able to do so frees up our minds to think about the real problem, which is understanding the real problem. I’m sure our customers will be happy to hear that the only way we have to practice that is to take their money and pretend that we know what we’re doing.

What are the causes of not knowing what we’re doing? I can think of at least two: not knowing what the requirements actually are (which includes knowing what everyone thinks they are but not knowing how fulfilling those will change them), and having an existing system to adapt to the new requirements. A lot of the time, both of these are unknown.

We already have plenty of “work with legacy code” katas. We call them open source issue trackers. Find a project, find a bug, dive in and fix it. Practice working with code that’s new to you, and already solves most of its problems.

You’ll find that with open source and free software, the two extremes of bugfixing get practiced a lot. At one end, people are willing to fix trivial bugs that mean inserting one if into the existing code. At the other end, people will do complete rewrites, throwing away working code and its bugfixes to introduce “clean”, though not necessarily working, code. It’s the expanse of space between these, where you need to make substantial changes but without throwing out the beneficial existing behaviour, where there’s space for legacy code katas.

But maybe we also need the “requirements kata”. Think of one of your interests that isn’t programming, and find a forum on it. Locate a thread where someone has asked a question that’s led to either no answer or a convoluted process to solve their problem, and think about how you could design a software system to solve that, and what other information you’d need to specify it well. Think about how you’d validate that the derived system actually solves the problem.

Posted in advancement of the self | Leave a comment

In The Design of Design, Fred Brooks makes an interesting point about ESR’s description of the Bazaar model of Linux (and, by extension, “Open Source”) development.

Linux was actually designed in a cathedral. The design was supplied by Unix, where Linux was to be a work-alike replacement for a particular component. There was even a functional specification: the GNU utilities already existed and the kernel had to support them.

Posted on by Graham | Leave a comment

An acceptable tool

It’s easy to forget that adequacy is, well, adequate. It’s easy to go all-in on making some code structure perfect, when good enough would be good enough.

Posted on by Graham | Leave a comment

Beware the IDEs

I recently had the opportunity to talk with a couple of software project managers from IBM. That company is of a kind that I have never worked at, and many of the companies I have worked at are of kinds that these IBMers have not worked at. There was thus plenty of opportunity for us to have different opinions and to explore those.

One such difference, though one we did not investigate in depth, is over IDEs. There was no doubt in the IBM view: a developer with an IDE is a more productive developer. If your programming team wants to license some proprietary IDE to use on your project, you’re probably better off paying for the licensing.

That’s not the world I come from, and indeed I may even have removed IDEs from my consciousness completely. Sure, I use a few, but what’s wrong with emacs, TAGS, and a bit of gud.el and M-x compile? I don’t mind launching an IDE when it’s there, but I don’t miss it when I work with separate tools.

So, are the people who write Rails apps in Sublime Text or vim doing it right, or are they missing out on clear productivity gains? More fundamentally, was it a good idea to build out something like Rails without keeping the IDE support in sync with the features?

There are all sorts of reasons to believe that IDEs are better.

  • IDEs are actually better, and IBM has a mature enough development system to be able to measure that.
  • IBM believes the Rational marketing (Rational’s price ticket was at the high end of software company acquisitions and perhaps IBM’s culture has, ahem, rationalised that).
  • There was a time that IDEs were better than developing without, but now bloatware text editors like emacs and vim have caught up without IBM’s view updating.

There are also reasons to believe that IDEs are not better.

  • IDEs are actually not betSourceKit Service crashed.
  • The cynic in me doesn’t believe the facts presented by the IDE vendors’ marketeers.
  • The bloatware text editors have actually caught up.
  • The productivity problems I have do not stem from my choice of editing environment.

What’s kindof interesting is to compare the views of dyed-in-the-wool IDE evangelists, dyed-in-the-wool text editor/command-line proponents, and fence-sitters and find out why they have different views, whether there are facts that can be teased out of these positions or assertions that can be validated. What’s really fascinating is general situation of which this is a single example: there are whole regions of the software development phase space that I have yet to experience.

Posted in tool-support | 2 Comments

Hiding behind messages

A problem I think about every so often is how to combine the software design practice of hiding implementations behind interfaces with the engineering practice of parallel execution. What are the trade-offs between making parallelism explicit and information hiding? Where are some lines that can be drawn?

Why do we need abstractions for this stuff, when the libraries we have already work? Those libraries make atomic features like threads and locks explicit, or they make change of control flow explicit, and those are things we can manage in one place for the benefit of the rest of our application. Nobody likes to look at the dispatch terrace:

    });
  });
});

Previous solutions have involved object confinement, where every object has its own context and runs its code there, and the command bus, where you ask for work to be done but don’t get to choose where.

Today’s solution is a bit more explicit, but not much. For every synchronous method, create an asynchronous version:

@interface MyObject : Awaitable

- (NSString *)expensiveCalculation;
- async_expensiveCalculation;

@end

@implementation MyObject

- (NSString *)expensiveCalculation
{
  sleep(5);
  return @"result!";
}

@end

int main(int argc, const char * argv[]) {
  @autoreleasepool {
    MyObject *o = [MyObject new];
    id asyncResult = [o async_expensiveCalculation];
    NSLog(@"working...");
    // you could explicitly wait for the calculation...
    NSLog(@"Result initial: %@", [[asyncResult await]   substringToIndex:1]);
    // ...but why bother?
    NSLog(@"Shouty result: %@", [asyncResult uppercaseString]);
  }
  return 0;
}

This is more of a fork-and-join approach than fire and forget. The calling thread carries on running until it actually needs the result of the calculation, at which point it waits (if necessary) for the callee to complete before continuing. It’ll be familiar to programmers on other platforms as async/await.

The implementation is – blah blah awesome power of the runtime – a forwardInvocation: method that looks for messages with the async marker, patches their selector and creates a proxy object to invoke them in the background. That proxy is then written into the original invocation as its return value. Not shown: a pretty straightforward category implementing -[NSInvocation copyWithZone:].

@implementation Awaitable

- (SEL)suppliedSelectorForMissingSelector:(SEL)aSelector
{
  NSString *selectorName = NSStringFromSelector(aSelector);
  NSString *realSelectorName = nil;
  if ([selectorName hasPrefix:@"async_"]) {
    realSelectorName = [selectorName substringFromIndex:6];
  }
  return NSSelectorFromString(realSelectorName);
}

- (NSMethodSignature *)methodSignatureForSelector:(SEL)aSelector
{
  NSMethodSignature *aSignature =   [super methodSignatureForSelector:aSelector];
  if (aSignature == nil) {
    aSignature =    [super methodSignatureForSelector:[self suppliedSelectorForMissingSelector:aSelector]];
  }
  return aSignature;
}

- (void)forwardInvocation:(NSInvocation *)anInvocation
{
  SEL trueSelector =    [self suppliedSelectorForMissingSelector:[anInvocation selector]];
  NSInvocation *cachedInvocation = [anInvocation copy];
  [cachedInvocation setSelector:trueSelector];
  CBox *box = [CBox cBoxWithInvocation:cachedInvocation];
  [anInvocation setReturnValue:&box];
}

@end

Why is the proxy object called CBox? No better reason than that I built this while reading a reflection on Concurrent Smalltalk where that’s the name of this object too.

@interface CBox : NSProxy

+ (instancetype)cBoxWithInvocation:(NSInvocation *)inv;
- await;

@end

@implementation CBox
{
  NSInvocation *invocation;
  NSOperationQueue *queue;
}

+ (instancetype)cBoxWithInvocation:(NSInvocation *)inv
{
  CBox *box = [[self alloc] init];
  box->invocation = [inv retain];
  box->queue = [NSOperationQueue new];
  NSInvocationOperation *op = [[NSInvocationOperation alloc]    initWithInvocation:inv];
  [box->queue addOperation:op];
  return [box autorelease];
}

- init
{
  return self;
}

- await
{
  [queue waitUntilAllOperationsAreFinished];
  id returnValue;
  [invocation getReturnValue:&returnValue];
  return [[returnValue retain] autorelease];
}

- (void)dealloc
{
  [queue release];
  [invocation release];
  [super dealloc];
}

- forwardingTargetForSelector:(SEL)aSelector
{
  return [self await];
}

@end

You don’t always need some huge library to clean things up. Here are about 70 lines of Objective-C that abstract an implementation of concurrent programming and stop my application code from having to interweave the distinct responsibilities of what it’s trying to do and how it’s trying to do it.

Posted in architecture of sorts, code-level, OOP | 4 Comments

It depends? It depends.

Sometimes you ask a question which has a small collection of actionable answers: yes or no. You ask someone who should be able to give that yes or no answer, and they go for the third: it depends.

Maybe they can’t actually answer your question, but want to sound profound.

Maybe they don’t realise that you’re a beginner in this particular area and would benefit from knowing what they’d do in this situation.

Maybe they don’t realise that you’re an expert in this area and are looking for confirmation or refutation of your existing choice.

Maybe they don’t realise that you’re uninterested in this area, and just need a decision made so you can move on.

Maybe they don’t know what the answer is, but are hoping you’ll help them think through the alternatives.

Sometimes, the answer to a question is “it depends”. But is that the answer you should give? It depends.

Posted in learning | Leave a comment