Blog | Structure and Interpretation of Computer Programmers | From programmer to software engineer.

Classes are globals, too

Posted on 2012-05-04 by Graham

Software engineers are used to the notion that global variables are a bad idea. Globals are usually accessed by asking, not by telling. They introduce tight coupling between any module that uses the global and the one that declares it, and (more dangerously) implicit coupling between all of the modules that use the global.

It can be hard to test code that uses globals. You typically need to supply a different object (by which I mean a linker object) that exports the same symbol, so that in tests you get to configure the global as you need it. Good luck if that’s deeply embedded in other code you do need, like NSApp.

While we’re on the subject of global variables, the names of classes are all in the global namespace. We know that: there is only one NSKeyedUnarchiver class, and everyone who uses the NSKeyedUnarchiver class gets the same behaviour. If any module that uses NSKeyedUnarchiver changes it (e.g. by swizzling a method on the metaclass), then every other client of NSKeyedUnarchiver gets the modified behaviour.

[If you want a different way of looking at this: Singletons are evil, and each class is the singleton instance of its metaclass, therefore class objects are evil.]

Now that makes it hard to test use of a class method: if I want to investigate how my code under test interacts with a class method, I have these options:

Swizzle the method. This is possible, but comparatively high effort and potentially complicated; should the swizzled method call through to the original or reimplement parts of it behaviour? What goes wrong if it doesn’t, and what needs to be supported if it does? Using the example of NSKeyedArchiver, if you needed to swizzle +archiveRootObject:toFile: you might need to ensure that the object graph still receives its -encodeWithCoder: messages – but then you might need to make sure that they send the correct data to the coder…
Tell the code under test what class to use, don’t let it ask. So you’d have a property on your object @property (nonatomic, strong) Class coderClass; and then you’d use the +[coderClass archiveRootObject:toFile:] method. In your app, you set the property to NSKeyedArchiver and in the tests, to whatever it is you need.
That’s simple, and solves the problem, but rarely seen. I think this is mainly because there’s no way to tell the Objective-C compiler “this variable represents a Class that’s NSCoder or one of its subclasses”, so it’s hard to convince the type-safety mechanism that you know archiverClass responds to +archiveRootObject:toFile:.
Use instance methods instead of class methods, and tell the code you’re testing what instance to use. This is very similar to the above solution, but means that you pass a fully-configured object into the code under test rather than a generic(-ish) class reference. In your code you’d have a property @property (nonatomic, strong) NSCoder *coder in to -encodeRootObject: and rely on it having been configured correctly to know what to do as a result of that.
This solution doesn’t suffer from the type-safety problem introduced in the previous solution; you told the compiler that you’re talking to an NSCoder (but not what specific type of NSCoder) so it knows you can do -encodeRootObject:.

Notice that even if you decide to go for the third solution here (the one I usually try to use) and use object instances instead of classes for all work, there’s one place where you must rely on a Class object: that’s to create instances of the class. Whether through “standard” messages like +alloc or +new, or conveniences like +arrayWithObjects:, you need a Class to create any other Objective-C object.

I’d like to turn the above observation into a guideline to reduce the proliferation of Class globals: an object’s class should only be used at the point of instantiation. Once you’ve made an object, pass it to the object that needs it, which should accept the most generic type available that declares the things that object needs. The generic type could be an abstract class like NSCoder or NSArray (notice that all array objects are created by the Foundation library, and application code never sees the specific class that was instantiated), or a protocol like UITableViewDataSource.

Now we’ve localised that pesky global variable to the smallest possible realm, where it can do the least damage.

Posted in code-level, software-engineering, TDD | Comments Off

Software-ICs and a component marketplace

Posted on 2012-04-28 by Graham

In the previous post, I was talking about Object-Oriented Programming, an Evolutionary Approach. What follows is a thought experiment based on that.

Chapter 6 of Brad Cox’s book, once he’s finished explaining how ObjC works (and who to buy it from), is concerned with his vision of how Object-Oriented software will be built. He envisions “Software-ICs”—compiled object files defining the code to support a single class (no need for header files, remember) that are distributed with documentation on how to use that class.

Developers or “software librarians” connect ICs together into collections called “categories”, which are implemented as object libraries. It’s a bit unfortunate that “category” is an inappropriate name mainly due to later reuse by NeXT; but then the alternate word “framework” is also unfortunate due to confusion with the computer science term (which allows that AppKit, WebObjects UIKit are frameworks, but Foundation, Quartz and so on are not). But it’s an entirely understandable reuse: in Smalltalk-80 related methods are grouped into categories, and NeXT used the same terminology for a very similar purpose.

Interestingly, Cox allowed for the compiler to generate vtables of selectors for each category, a bit like the Amiga operating system’s library format. That’s to support having different variable types for selectors with the same name. Modern Objective-C doesn’t support that; if you define selectors with the same name but different parameters or return values, you’ll get a warning and your code might not work correctly.

Finally, an application is a network of categories connected by the linker. One (or perhaps more, depending on your design) of the categories in the application contains the app-specific classes.

My reason for bringing this up is that this vision of object-oriented software engineering closely models component-oriented hardware engineering by allowing for software shops to produce catalogs of the components they produce at each level. Just as you can order a single IC, or a circuit board with a few ICs connected, or a whole widget, so you could order a class, or a category, or an application. If you want to build a new application, you might buy a couple of classes from one vendor, a category from another vendor, then write a few classes yourself and integrate the whole lot.

Enough ancient book, talk about the real world.

We have a lot of this, and make quite a lot of use of it. There are loads of Objective-C classes, libraries and frameworks out there for us to use, and to some extent there are catalogs. Many of the components we can use are open source, which means that we can treat the class interfaces themselves as the catalogs. If we’re lucky there’ll be some documentation, perhaps in the form of AppleDoc or a README.

Unfortunately availability vastly outstrips discoverability. You have to go to multiple catalogs to ensure that you’ve exhausted the search space: Google Code, GitHub, BitBucket, SourceForge etc. in addition to finding commercial libraries which won’t be listed in any of those places. Actual code search engines like OpenGrok and Koders are great for finding out about source code, but not so great for discovering it in the first place.

Metacatalogs like Cocoa Objects, Cocoa Controls and CocoaOpen solve part of this problem by letting people list their source code in a single place, but because they’re incomplete they only add to the number of places you need to search.

Then, once you’ve got the component, what do you do? Are you meant to drop the source files into your project? Should you drop the project in and add the library as a dependency of your app? Should you use CocoaPods?

Learn from what we already do

Just as we already push most of the apps we write to a single app store where customers can discover, purchase and install apps in a state where they’re ready to use, we should do the same with components.

[Please bear in mind that like most descriptions of ideas, a lot of nuances and complexity are known but are elided below for the sake of clarity. Comment brownie points will not be awarded for comments that explain how I haven’t considered case X; I probably have.]

A component store would, for browsers, start off very similar to Cox’s idea of a component catalog. You’d go to it and search for a component that suits your needs. You could see a “spec sheet” for each component detailing what it does, what it costs, the terms of using it and that sort of thing. You’d then buy the component if it’s paid for and download it. If the licence permits it you could download the source, too.

The download would drop the binary and headers into a folder that Xcode would recognise as an additional SDK. It would also drop the documentation in docset format into a standard location. An Xcode project would just need to point at the additional SDK and it could pick up all of the components available to the developer.

From the perspective of a component manufacturer, the component store would look a little like iTunes Connect. You’d write your code, then package it up for the store in a standard way along with the description that goes into the “spec sheet”. For open source projects that could just involve git push componentstore master to have the store itself generate the binaries and documentation from the source code.

Posted in Business, code-level, OOP | Comments Off

Comparing Objective-C and Objective-C with Objective-C

Posted on 2012-04-28 by Graham

A while back, I wrote an object-oriented dispatch system for Objective-C. It defines only three things: an object type (the BlockObject), a way to create new objects (the BlockConstructor), and a way to message objects (the dispatch mechanism).

That’s all that the first version of Objective-C defines, as documented in Brad Cox’s Object-Oriented Programming: an Evolutionary Approach.

Objective-C is a hybrid programming language[…]formed by grafting the Smalltalk-80 style of object-oriented programming onto a C language rootstock. Objective-C adds precisely one new data type, the object, to those C provides already, and precisely one new operation, the message expression. Like Smalltalk-80, Objective-C makes no compile-time distinction between different kinds (classes) of objects.

So if you want to try out 1980s-style OOP using Objective-C, the tool to use is not Objective-C itself but BlockObject.

That quote is from the start of Chapter Four (of the Cox edition, not the Cox and Novolbilski edition), which goes on to describe the Objective-C preprocessor, the multiple-dispatch system (using strings as message selectors like BlockObject does—and indeed like Apple’s ObjC does albeit hidden behind the opaque SEL type), and the _msg C function that implements message dispatch.

Factory Objects

Having objects is all very well, but we need some way to create them. In the BlockObject system, there’s a special type called BlockConstructor that configures and returns a new instance of an object.

That’s boring. Objective-C doesn’t define a “constructor type”, can’t we just use objects? Well, yes. Given a class definition like this:

= Stack : Object { id *objects; unsigned int count; }

Objective-C as-was automatically defines a global factory object called Stack that can be used anywhere in the code. You grab it and use it like this:

extern id Stack;

id myStack = [Stack new];
[myStack push: someObject];

From there, Objective-C won’t surprise you much. You can define instance methods:

-push:anObject {
	if (count == MAX_OBJECTS) [self cannotGrow];
	objects[count++] = anObject;
	return self;
}

and factory methods:

+new {
	id stack = [super new];
	stack->objects = malloc(MAX_OBJECTS * sizeof(id));
	return stack;
}

Though users of “modern” Objective-C will notice that there’s no compile-time type checking: no Stack *stack variable declaration for example. Indeed there’s no @interface for the class at all; all objects are of type id, you can send any message to any object and what it does with that message is up to the receiver. Just like Smalltalk or Ruby.

Posted in code-level, OOP, software-engineering | Comments Off

The debugger of royalty

Posted on 2012-04-27 by Graham

We’ve all got little libraries of code or scripts that help us with debugging. Often these are for logging information in a particular way, or wrapping logs/tests such that they’re only invoked in Debug builds but not in production. Or they clean up your IDE’s brainfarts.

Having created these debug libraries, how are you going to get your production code to use them? Are you really going to sprinkle MyCompanyDebugLog(fmt,…) messages throughout your app?

Introducing step one on the road to sanity: the debug proxy. This is useful when you want to find out how a particular class gets used, e.g. when it provides callbacks that will be invoked by a framework. You can intercept all the messages to the object, and inspect them as you see fit. Here’s the code (written with the assumption that ARC is enabled):

FZADebugProxy.h

#import <Foundation/Foundation.h>

@interface FZADebugProxy : NSProxy

- (id)initWithTarget: (NSObject *)aTarget;

@end

FZADebugProxy.m

#import "FZADebugProxy.h"

@implementation FZADebugProxy {
    NSObject *target;
}

- (id)initWithTarget:(NSObject *)aTarget {
    target = aTarget;
    return self;
}

- (NSMethodSignature *)methodSignatureForSelector:(SEL)sel {
    NSMethodSignature *signature = [target methodSignatureForSelector: sel];
    if (signature == nil) {
        signature = [super methodSignatureForSelector: sel];
    }
    return signature;
}

- (BOOL)respondsToSelector:(SEL)aSelector {
    return [target respondsToSelector: aSelector] ? YES : [super respondsToSelector: aSelector];
}

- (void)forwardInvocation:(NSInvocation *)invocation {
    invocation.target = target;
    SEL aSelector = [invocation selector];
    (void)aSelector;
    [invocation invoke];
}

@end

And no, there isn’t a bug in the -initWithTarget: method. The slightly clumsy extraction of the selector in -forwardInvocation: is done to avoid a common problem with using Objective-C inside the debugger where it decides it doesn’t know the return type of objc_msgSend() and refuses to call the method.

You would use it like this. Here, I’ve modified the app delegate from BrowseOverflow to use a proxy object for the object configuration – a sort of domain-specific IoC container.

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
    BrowseOverflowViewController *firstViewController = [[BrowseOverflowViewController alloc] initWithNibName: nil bundle: nil];
    firstViewController.objectConfiguration = (BrowseOverflowObjectConfiguration *)[[FZADebugProxy alloc] initWithTarget: [[BrowseOverflowObjectConfiguration alloc] init]];
    TopicTableDataSource *dataSource = [[TopicTableDataSource alloc] init];
    [dataSource setTopics: [self topics]];
    firstViewController.dataSource = dataSource;
    self.navigationController.viewControllers = [NSArray arrayWithObject: firstViewController];
    self.window.rootViewController = self.navigationController;
    [self.window makeKeyAndVisible];
    return YES;
}

The bold line is the important change. The cast silences the compiler’s strict type-checking when it comes to property assignment, because it doesn’t believe that NSProxy is of the correct type. Remember this is only debug code that you’re not going to commit: you could switch to a plain old setter, suppress the warning using diagnostic pragmas or do whatever you want here.

At this point, it’s worth running the unit tests and using the app to convince yourself that the behaviour hasn’t changed at all.

So, how do you use it? Shouldn’t there be an NSLog() or something in the proxy class so you can see when the target’s messaged?

No.

Step two on the road to sanity is to avoid printf()-based debugging in all of its forms. What you want to do here is to use Xcode’s debugger actions so that you don’t hard-code your debugging inspection capabilities into your source code.

Set a breakpoint in -[FZADebugProxy forwardInvocation:]. This breakpoint will be met whenever the target object is messaged. Now right-click on the breakpoint marker in the Xcode source editor’s gutter and choose “Edit Breakpoint…” to bring up this popover.

Xcode's breakpoint popover.

In this case, I’ve set the breakpoint to log the selector that was invoked, and crucially to continue after evaluation so that my app doesn’t stop in the debugger every time the target object is messaged. After a bit of a play with the app in the simulator, the debug log looks like this:

GNU gdb 6.3.50-20050815 (Apple version gdb-1752) (Sat Jan 28 03:02:46 UTC 2012)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin".sharedlibrary apply-load-rules all
Attaching to process 3898.
Pending breakpoint 1 - ""FZADebugProxy.m":37" resolved
Current language:  auto; currently objective-c
0x10271:	 "stackOverflowManager"
0x102a0:	 "avatarStore"
0x10271:	 "stackOverflowManager"
0x10271:	 "stackOverflowManager"
0x102a0:	 "avatarStore"

Pretty nifty, yes? You can do a lot with Xcode’s breakpoint actions: running a shell script or an AppleScript are interesting options (you could have Xcode send you an iMessage every time it send your target an Objective-C message). Speaking out the names of selectors is fun for a very short while, but not overly useful.

Xcode’s breakpoint actions give you a much more powerful debugging capability than NSLog(). By using breakpoint actions on an Objective-C proxy object, you can create highly customisable aspect-oriented techniques for debugging your code.

Posted in code-level, iPad, iPhone, Mac, TDiOSD | 11 Comments

TDD and crypto in one place

Posted on 2012-04-19 by Graham

Well, I suppose if I’ve written two books, it’s about time I wrote a contorted blog post that references both of the worlds.

I recently wrote an encryption module for an app, and thought it’d be useful to share something about the design process. Notice that the source code here was quickly thrown together for the purposes of demonstrating the design technique, and bears little resemblance to the code I was actually writing (which was in a different language). A lot of the complexity I was actually trying to deal with has been elided in order to bring the story to the fore.

Steps 1a, 2a, 1b, 2b etc: Design the API hand-in-hand with a fake implementation.

A pass-thru implementation would work here, but I prefer to use something that actually does change its input just so that it’s clear that there’s a requirement the input should be acted on. You might end up with tests that look something like this:

@implementation CryptoDesignTests
{
    id <StringEncryptor>cryptor;
}

- (void)setUp {
    cryptor = [[ROT13StringEncryptor alloc] init];
}

- (void)tearDown {
    cryptor = nil;
}

- (void)testEncryptionOfPlainText {
    NSString *plainText = @"hello";
    NSString *cipherText = [cryptor encipherString: plainText];
    STAssertEqualObjects(@"uryyb", cipherText, @"This method should encrypt its parameter");
}

- (void)testDecryptionOfCipherText {
    NSString *cipherText = @"uryyb";
    NSString *plainText = [cryptor decipherString: cipherText];
    STAssertEqualObjects(@"hello", plainText, @"This method should decrypt its parameter");
}

@end

Where the protocol looks like this:

@protocol StringEncryptor <NSObject>

- (NSString *)encipherString: (NSString *)plainText;
- (NSString *)decipherString: (NSString *)cipherText;

@end

Of course you can implement this however you want. Here’s one potential implementation (which may have its problems, but is only being used to guide the API design):

@interface ROT13StringEncryptor : NSObject <StringEncryptor>

@end

@implementation ROT13StringEncryptor

- (NSString *)rot13: (NSString *)original {
    NSMutableData *originalBytes = [[original dataUsingEncoding: NSASCIIStringEncoding] mutableCopy];
    for (NSInteger i = 0; i < [originalBytes length]; i++) {
        NSRange cursor = NSMakeRange(i, 1);
        char c;
        [originalBytes getBytes: &c range: cursor];
        if       (c >= 'a' && c <= 'm') c += 13;
        else if  (c >= 'n' && c <= 'z') c -= 13;
        else if  (c >= 'A' && c <= 'M') c += 13;
        else if  (c >= 'A' && c <= 'Z') c -= 13;
        [originalBytes replaceBytesInRange: cursor withBytes: &c];
    }
    return [NSString stringWithCString: [originalBytes bytes] encoding: NSASCIIStringEncoding];
}

- (NSString *)encipherString:(NSString *)plainText {
    return [self rot13: plainText];
}

- (NSString *)decipherString:(NSString *)cipherText {
    return [self rot13: cipherText];
}

@end

Step 3. Write an integration test.

Having designed the API for the class, I'd like to write an integration test that does whatever it is that the app is trying to achieve. In my real app this task was that two cryptors hooked up to each other should agree on a key and pass a message between each other. For this demo, we'll require that the cryptor can round-trip a message and end up with the original text.

@implementation CryptoRoundTripTests
{
    id <StringEncryptor>> cryptor;
}

- (void)setUp {
    cryptor = [[ROT13StringEncryptor alloc] init];
}

- (void)tearDown {
    cryptor = nil;
}

- (void)testRoundTripEncryption {
    NSString *plain = @"Mary had a little lamb.";
    NSString *outOfTheSausageMachine = [cryptor decipherString: [cryptor encipherString: plain]];
    STAssertEqualObjects(plain, outOfTheSausageMachine, @"Content should survive an encryption round-trip");
}

@end

This test passes without any extra work.

Step 4. Give that lot to the poor sap who needs to write the app.

That's right, the other developers can start working straight away. They've got the protocol which tells them what the object can do, the unit tests which explain how the API should work and the integration test that explains how you expect the object to be used.

Leaving you free to:

Step 5. Point the integration test at a different implementation of the protocol.

But you haven't written that implementation yet! It must be time to:

Step 6. Write it.

You know when it works because you have integration tests. You know it'll work with the app the other person's writing because they've got the same integration test.

So the extra artefacts that some people see as a wasteful byproduct of code-level testing — the fake implementation of the protocol for example — are demonstrably useful in this scenario. Other developers on the team can use the fake implementation as a stand-in while you're head-down coding up the production code. It's also much easier to design a class when you're not also sweating over how the internal details are going to work out, so you probably get where you're going quicker too.

Posted in code-level, Crypto, TDD | Comments Off

Culture, heritage and apps

Posted on 2012-04-15 by Graham

I said earlier on Twitter that I’m disappointed with the state of apps produced for museums and libraries. I’d better explain what I mean. Here’s what I said:

Disappointed to find that many museum apps (British Library, Bodleian, Concorde etc) are just the same app with different content. :-(

In each case (particularly Concorde) there’s some magic specific to the subject. Rubber-stamping the app doesn’t capture that magic.

They’re all made with a tool called Toura that sits atop PhoneGap. Makes it easy to get the content into the apps at the cost of expression.

So to be clear: my problem is not with the museums and other heritage sites. I’m familiar with the financial and political problems associated with running a museum. My social circle includes curators, librarians and medieval manuscript experts and I know that money is tight, that oversight is close and that any form of expenditure in promotion must result in a demonstrable increase in feet through the door and donations to be considered a success.

The Magna Carta

My problem is also not with the Toura product and the team behind it. They’re to be commended for identifying that while the culture and heritage community doesn’t have much money, they still deserve to be represented in our tablets and phones. The apps I listed above all feature astounding objects: examples include the Concorde aircraft, the Lindisfarne gospels and the Magna Carta.

Concorde

So why are these apps disappointing? It’s basically because they’re all the same. Each of these objects has its own magic: the unique shape of Concorde, the vivid colours of the gospels and the constitution-defining text of the Carta. So why present them in the same way? Why not make a Concorde app that evokes aerodynamic speed, an illuminated Lindisfarne gospels app, and a revolutionary Magna Carta app? As many people have explained, it’s because standard content-viewing apps like Toura are all these institutions can afford.

Folio 27r from the Lindisfarne Gospels: image from Wikimedia Commons

There’s something seriously wrong in our app industry. We’ve created a world in which apps are too cheap, and developers struggle to make a living from 70¢ per sale. Simultaneously, we’ve created a world in which apps are too expensive: the keepers of some of the world’s most interesting objects can’t afford to showcase these with more than a generic master/detail view. What’s that about?

What mindset leads us to demand high worth, when we’re making products that we can’t convince customers contain any value? If they don’t see the value, are we deluding ourselves? People talk about events like the Instagram acquisition as being evidence of a bubble subset of the app economy. But isn’t that oxymoron – a free app that’s worth a billion dollars – just an exaggerated version of what the rest of the industry is up to?

Let me present this problem in a different way, by moving from apps about museums to museums about apps.

The Rijksmuseum, Amsterdam.

Last month I spent an enjoyable time with some very good friends in Amsterdam, the city of both apps and museums. While walking around the Rijksmuseum, I reminded myself that we’re in the middle of the possibly the largest and definitely the fastest technology-mediated social revolution humanity has ever experienced. Those of us enabling the application of this technology are literally providing the fulcrum around which our species is revolving.

During this year, museums will look at apps as cultural, historical and ethnological artefacts in their own right. Over the coming century, heritage institutions will shape the way that our brief period in time is presented for posterity. Be sure that all of the million apps currently available across all app stores will not be on display.

As with any presentation, perfection is achieved not when there is nothing left to add but when there is nothing left to take away. Just as you can’t (and wouldn’t want to) walk into a museum and see every manuscript from the 10th century on display, you won’t be able to walk into the Nieuwerijksmuseum in 3112 and see every app from the 21st century. Only those that are considered cultural treasures will be given pride of place in the galleries. Can something that apparently has no value be considered a cultural treasure? Will any of your apps be part of the presentation? I don’t think I’ve yet produced anything that will be, and that needs to change.

Posted in Business | 2 Comments

Test-Driven iOS Development

Posted on 2012-04-12 by Graham

Here it is, after more than a year in the making, the book that they really did want you to read! Test-driven IOS Development (Developer’s Library) (affiliate link) has finally hit the stores[*].

I wrote this book for the simple reason that it didn’t exist. Like Professional Cocoa Application Security (Wrox Professional Guides) (another affiliate link), I knew that the topic was something many people were interested in, including myself. I wanted a single place I could go to find out about using Xcode for writing unit tests, and how the Test-Driven Development approach could be applied to writing Cocoa code with Objective-C.

Well, it turned out that this place didn’t exist, so I tried my best to create it. In the book you’ll see how the BrowseOverflow app was built entirely in a test-first fashion, using OCUnit and Xcode 4. Along the way, I hope you’ll see how TDD can be useful for your own projects.

If you’re interested in finding out more about test-driven development for iOS apps, I’ll be giving a talk on the subject at iOSDev UK in Aberystwyth in July. And of course I’m happy to field questions about the book or about TDD here or over on Twitter. And of course my unit testing video training course is still available from iDeveloper.tv.

Happy testing!

[*]The availability appears to vary by country and format, so if it isn’t in your favourite (e)bookstore yet just be patient and keep refreshing that page like it’s the WWDC site.

Posted in books, PCAS, software-engineering, TDD, TDiOSD | 6 Comments

On the magic of key agreement

Posted on 2012-04-12 by Graham

Imagine that you want to implement AirDrop, or something like it. Two computers that have (possibly) never communicated before are going to share a file. Now you know that you want to encrypt the file in transit so that only these two computers get to see the file, but how do you set up an encryption key? You can’t generate it on one of the computers and send it over, because that way lies the infinite stack of turtles: how do you keep the key secure in transit?

The solution (or at least, one solution known as the Diffie–Hellman protocol) turns out to be beautifully simple. First, arrange that everyone knows two numbers, which we’ll call the base or generator g and the modulus m. For example, in the SKIP protocol these two numbers are documented:

g = 0x cf ca d9 90 8d 98 c1 c4 4e 4e 88 20 69 e8 7b fb
d0 ca e1 ee 24 64 a9 6c 89 70 a0 8c 86 00 30 c6
c2 83 4a 0d 82 bd a5 ba 10 cf 80 af 61 96 11 4b
3b 87 a9 7e 59 98 c4 aa 0c bd b7 2b 23 18 33 64
f3 b6 2e 1a 4d e8 86 85 aa de 78 a1 5d ce 65 33
75 7a 9d 00 b9 9e 05 26 ed 79 62 15 97 c4 06 26
fa 51 e1 5e f5 1d cb d2 23 e2 73 e5 f2 c7 3c c4
de 58 cd 3b b6 15 3c aa 84 7c fa 5f cb 6b 8d 78

m = 0x dd c9 08 3e 8f ae ef 28 16 ad 50 a9 68 ac bd 04
8e 90 9e ab 5d 41 fc 0a 51 3f 86 0a c4 1b 22 b9
30 dc a3 0a 59 73 05 38 59 b7 85 66 df ff c6 5b
b9 1f fe 44 d9 d6 5e cb 9b 68 38 a1 fd 25 3f 01
51 88 9e 93 c3 22 24 f9 03 e6 9b 8b 07 34 9d 9f
9b 38 4b c0 97 03 dc e5 2e 92 47 4c 2c e1 59 26
14 82 49 dd 58 13 91 05 12 11 a1 45 06 ac 11 8f
b1 83 53 46 93 88 9a 46 b1 8a 01 50 cb 5e 82 55

The two computers each generate a random number, which we’ll call x_A and x_B. They keep these numbers secret, but can independently calculate:

r = g^(x_A) mod m

s = g^(x_B) mod m

and they send each other r and s. The first computer can use x_A and s to derive:

k_A = s^(x_A) mod m = g^(x_A.x_B) mod m

while the second uses x_B and r to derive:

k_B = r^(x_B) mod m = g^(x_A.x_B) mod m ≡ k_A

They got the same key! Because you can’t gain any of the secrets k, x_A or x_B knowing only r and/or s, the two systems can negotiate a secret key that’s only known to the two of them without having to first establish a secure channel. I just think that’s really cool.

If you want to use this stuff in an iOS app, OpenSSL contains an implementation of Diffie–Hellman. On Android javax.crypto.KeyAgreement can do it for you.

Posted in Crypto | Comments Off

On my newer competence

Posted on 2012-04-04 by Graham

This time last year, I evaluated myself against the programmer competency matrix. So where am I one turn around the daystar later? I have to admit that this was mainly because I was jet lagged in a hotel room in Seattle and had nothing else to do last year :-), but nonetheless it’s time to reflect on how things have gone.

Last year’s goals

I wrote:

it looks like my priorities for research goals over the upcoming year or so should be:

Improve breadth and depth of API knowledge

Investigate systems decomposition of a smartphone/desktop/internet system.

Learn how to create and support DSLs

Write and publish improved developer support tools

So how did that go?

I improved here. I didn’t have a project for doing this, I just used the APIs some more. Things that I previously remembered but needed to look up I now automatically get right, and some things I didn’t know about I now do. There’ll always be room for more.
My work on GNUstep web is only part of this goal.
I haven’t done anything with DSLs, as I haven’t found a need for them in the last year. Maybe that’s because I don’t understand where they can be used :). Part of my GNUstep web project will include rules engines a la Direct 2 Web, so I guess I’ll have to tackle this soon.
Again, in hand with the GSW project. I’ve already published a code generation tool. Some of the things I learned about at NSConference were very inspirational.

What about this year?

I’ll just dive into the matrix. If nothing’s changed from last year you won’t see any comments.

Computer Science

Data Structures: still level 1. I’ve learned a little more about level 2 aspects.
Algorithms: level 2.
Systems programming: level 3, if you’re willing to accept that block-diagram-level knowledge counts as knowledge on the level chart :)

Software Engineering

Source Control: level 3.
Build Automation: level 3.
Automated Testing: level 3.

Programming

Problem Decomposition: level 3.
Systems Decomposition: level 3. As I said earlier, I made this a specific goal of the last year, and I believe it’s paid off.
Communication: level 3. I’ve also done what I set out to do last year, and dialled back on the commitments to conferences and books.
Code Organisation Within a File: level 3. As I said last year, this is subjective.

Code Organisation Across Files: level 3.

Source Tree Organisation: level 3. For these two I’ve decided it’s what you see in the IDE (Xcode, AppCode and Eclipse are my regular haunts) that really matters, though I try to make the filesystem layout match that.

Code Readability: level 3. I’m not quite that bastard who rejects code because it doesn’t conform to the standard; though I used to be :-(

Defensive Coding: level 2. Still don’t have a “library” for doing that (I’ve written an iOS security library that’s internal to O2, but that’s not what this is about.)

Error Handling: level 3 (with s/xception/rror/, because we appreciate that error-handling should be sympathetic to the platform/framework style, just like other code.)

IDE: level 3. It was mainly working with AppCode that spurred me into investigating ways to automate IDE functions.

API: still going to claim level 1, because I’d say it’s for other people to decide whether they think I have sufficiently “vast and in-depth knowledge”.

Frameworks: level 2.

Requirements: level 3.

Scripting: level 3 (though this improvement only just scraped in this year.)

Database: level 1. I’m starting to see more projects where a better knowledge of databases is required, if one of those lands in front of me I’d better improve at knowing databases. However as I said last year it’s the kind of thing where I can use an ORM to get it mostly correct, then investigate whether any issues remain that require deeper inspection.

Experience

I skipped this section last year, but just for kicks I’ll go through it this time. No comparisons with last year, and if you can’t work out why you’re not allowed higher than Level 0 on any entry in the matrix ;-).

Languages with Professional Experience: level 2. Functional programming I have worked with includes LISP, C++ and Java (yes, that Java).
Platforms with Professional Experience: heh, this depends as I said in a recent talk on how you define “Unix”. Conservatively I could say Unix, Mac, iOS and Java: level 1.
Years of Professional Experience: not that I think this matters, but level 2.
Domain Knowledge: wait, which domain? In the world of telco (where I currently spend my time) I’m a complete noob. In fact I’ve spent most of my career as a complete noob. Whether that’s good or bad, and something I should or would want to change, is open for debate. Maybe that’s another blog post, later.

Yeah, having written that section I now understand why I left it out last time.

Knowledge

Tool Knowledge: still level 2. I’ve written and published some codeine scripts, not sure it’s enough to qualify at level 3.
Languages Exposed to: still level 2. I did some concurrent programming in Objective-C, but no Erlang, Oz or Prolog. OK, I have done some Prolog tutorials, and Oz is the operating system on my Cambridge Z88 but I’m going to assume that it’s a different Oz.
Codebase knowledge: hard to gauge. We have about 100 repos on my team, some I’m not working on, some I collaborate on, some I’m the sole author. We’ll call this one unknowable for now.
Knowledge of Upcoming Technologies: level 3.

Platform Internals: level 2.

Books: still level 1, but I’ve written two now. I don’t imagine writing any more, there must be a better way to share knowledge.

Blogs: level 3.

This year’s goals

Some of the things I planned for last year still need to be done:

DSLs
Developer tools

To which I’ll add some things that, probably, don’t directly follow from the competency matrix, but which are nonetheless goals of mine.

More do, less talk. A side-effect of being “the security guy” in my corner of the industry is that I do a lot of talking about security in other peoples’ products, and not much making products. It’s time to find a product I believe in, and own the crap out of it. That means investing my time in it long-term, not diving in and out like a consultant. Consultants are great and consultancy is great: but I want to build something.
Mentor new developers. A recurring theme from my discussions in Appsterdam and at NSConf is that while I can certainly do a lot to become a better programmer, that won’t scale as well as making a load of other people better programmers. If I can help at least one person become a better developer than I am, I’ll improve my own abilities along the way and then there’ll be two people better than I am.

That will, I think, do for now.

Posted in Business, software-engineering | Comments Off

An apology and an opportunity

Posted on 2012-04-01 by Graham

Today’s earlier post, UX is snake-oil bullshit, was indeed an April Fool. Sorry to the people who had their “WTF blood boil”, among other reactions.

I’m also sorry to the people I parodied in the post. Please feel comfortable knowing that if I didn’t respect you and what you did, I wouldn’t feel willing to send you up in that fashion.

Now of course some people saw the joke, and some people didn’t. But what was most interesting was that in both camps there was a (large) fraction of respondents who recognised in the jokes the germ of truth. They realised that the humour came not from eschewing reality completely, but from taking cues from it and dialling them up to 11 so that the absurdity became clear.

So each of the points in my previous post is true, just not as true as the rhetoric would have you believe. The field of UX does contain people who can spout the platitudes, but don’t back it up. There is an amount of “look at this fail”. It is a fast-moving field, so it’s hard to find timeless, consistent, actionable advice. And there are people who will tell you that whatever they’re doing (or selling) is UX, that when bolted to your product creates magical UX goodness.

What that means is not, as my April Fool post concluded, that it’s time to withdraw from study of User Experience. It means that there’s a tremendous appetite among software engineers for resources offering specific, actionable tasks that we can undertake to improve the experience of our products.

One of the books on user experience I’ve got the most out of is Don’t Make Me Think!: A Common Sense Approach to Web Usability by Steve Krug. When he does have some specific issue to point out in the design of a website, he also shows either how the problem was or could be addressed. We could do with a “Don’t make me think!” for the mobile user experience. We need someone to cherry-pick the research on user testing, ethnography and other techniques, telling us what’s important and what to ignore. What we can be doing, and what we should be doing.

That’s a great opportunity for someone who wants to become the Smartphone UX Boffin.

Posted in user-error | 1 Comment

Structure and Interpretation of Computer Programmers