Structure and Interpretation of Computer Programmers

I make it easier and faster for you to write high-quality software.

Friday, March 30, 2012

Automate all the server Objective-C!

I decided it was time to stop writing WebObjects/GNUstepWeb code, and write some code that would make it easier to write WO/GSW code. With that in mind I replaced my previous component generator with a more robust generator.

I also wrote and published some git hooks for working on these projects. The pre-commit hook just runs ‘make’ and doesn’t let you commit if you can’t build: we’ll look at testing in a later post (you may not know this, but I’ve done a thing or two with Objective-C unit tests).

The post-commit hook launches the direct connect app, so you can have a box that’s always running the latest version for testing. You’d want to do something similar, though really not the same, for a production box: as well as being sensitive to database and web server adaptor[*] configurations, you’d want to be stricter about when you restart the server, and may have some app-specific work to do like triggering cleanup code. Besides which, it makes more sense to do that in a post-update hook.

[*] While a GSW app does include a web server, it’s common to restrict access to that server just to the internal network. Externally you have a “normal” web server like Apache or Nginx, with an adaptor that knows how to parse GSW URLs and redirect requests to the correct application.

posted by Graham at 00:15  

Wednesday, March 28, 2012

Adding components to a GNUstep web / WebObjects app

In WebObjects, Components take the role of a view controller in what passes for Cocoa’s version of MVC. Each is responsible for calculating the data that the view objects are bound to: you saw an example of this in the previous post. Each is also responsible for responding to user interface events—called actions—and preparing the next component where that’s required.

Navigating between components

Let’s add a new component to our HelloGSW test app. We’ve recorded the date that the visitor arrived at the site, we could (for sake of demonstration, not because it’s useful) find out how long it takes that visitor to get to the next page. We’ll come onto the mechanics of adding it to the project later, but for now we note that we require a way for the visitor to indicate their intention to see the next page. That means, as a simple way to do things, having a form with a submit button.

We could write the form into the HTML, but then we’d have to know what URL the WO machinery was going to use to call our action. So we’ll use WebObjects to generate the form, using this HTML:

 <webobject name=form>
   <webobject name=submit_button><INPUT TYPE="SUBMIT"></webobject>

As you’ve seen, the data bindings for each webobject need to be defined in the WOD file:

form:WOForm {}
submit_button:WOSubmitButton {action = nextPage}

As with the value binding you saw last time, we can bind the action of the submit button to a method on the component. Those of you who have used Cocoa Bindings may be surprised to find that you can do the same with NSResponder subclasses, though it’s rare to see it done with Mac apps.


- (GSWComponent *)nextPage {
  ClickDelay *nextComponent = (ClickDelay *)[self pageWithName: @"ClickDelay"];
  nextComponent.startDate = accessedAt;
  return nextComponent;

As with view controllers on the iPhone, you just create the new component (-pageWithName: does that, and configures its context correctly) and configure state where appropriate. Unlike with iOS view controllers, you don’t specify how they get presented because there’s only one way that will (probably) ever happen: in an HTTP response. So you can just return the next component from the method and the Right Thing™ will happen. If you return nil, the visitor will see the current component again (though with a whole server round-trip and a fresh HTML response, so it’s best to do some AJAXey thing: on which more later!).

If you’re really interested in how the other component works, check out its source from GitHub.

But that’s not why we’re here.

Adding a component involves a boatload of boilerplate: five new files, and a change to the GNUmakefile. That’s one more boatload than you’d like to write every time you create a new page in your web app, yes?

Yes. Therefore what I actually did was to write a script to do all of that boilerplate for me. It’s really basic (error checking is for security boffins. Oh, wait.) but will do the necessary dance. Just be sure not to run it twice for the same component yet…

What I really need is a tool that’s both more robust, and that can generate the other things you need in writing a WO app. I’ll start right now, but pull requests are always welcome…

posted by Graham at 23:05  

Sunday, March 25, 2012

Using Objective-C on the server

My talk at NSConf was about cross-platform Objective-C. Those people who I talked to after the session will know that my goal is clear. There are plenty of people out there who have learned Objective-C to get onto the iOS or Mac app stores. Those people need some server-side smarts in their apps, and want to benefit from the skills they’ve already acquired. In other words: they want Objective-C on the server.

I’m now in a position to offer that :). I took a look at the GNUstepWeb project, a reimplementation of WebObjects 4.5. It was in a fairly parlous state, and wouldn’t even build on my Mac. I’ve fixed that. It’s still a bit of a faff, but we’re in a position to start playing with server-side ObjC and see whether there’s any value in having a better product.

Building GSW on the Mac


First, you need to grab GNUstep from SVN. Now you need to compile GNUstep-make, the build system. On Mac OS X you do the following:

cd modules/core/make
./configure --with-library-combo=apple-apple-apple
make install

The weird configure invocation tells GNUstep you’re going to use the Apple Objective-C runtime, and the Foundation (and AppKit) libraries instead of GNU replacements.

GNUstep-make includes a shell script that defines the locations of various folders. You should import that into your shell environment:

. /Library/GNUstep/Makefiles/

Consider adding that line to your .bash_profile.


Then you build the GNUstep-base extensions, some additional classes and macros for working with Foundation code.

cd modules/core/base
make install


GDL2 is the GNUstep Database Library; an EOF-like framework for interacting with databases. If you’re going to use a DBMS like MySQL for your web apps, you should install that and its developer headers before doing:

cd modules/dev-libs/gdl2
./configure --disable-gorm-palette
make install

(Aside: GORM is the GNUstep version of Interface Builder. You don’t need the plugin for working with entities in GORM, and you can’t build it on Mac OS X so that’s why it’s disabled in the configure invocation.)


Now here’s the good stuff. Build and install GSW. At time of writing, there’s an open bug that stops it building on Mac OS X, so you’ll need to grab the patch from this bug report. Apply that patch, then:

cd modules/dev-libs/gsweb
make install

A sample app

You probably want to see a sample GSW app, so it’s a good job I wrote one :-). Clone the HelloGSW project on GitHub. I’ll show you how to build and run that before we get into how it works.

Building HelloGSW

cd HelloGSW

Running HelloGSW


this will spit a load of output to the console, including a URL that looks like http://localhost:9001/WebObjects/HelloGSW.woa/0/. Open that URL in your browser. You should see this:

Hello from GSW

HelloGSW code walkthrough

The interesting code is in the Main class, which is a subclass of GSWComponent (and conventionally the default component to show for a new visitor). Each “page” in a GSW app is a component, though of course you could arrange to return XML, JSON or any other format rather than HTML. HTML is the default and the easiest to work with, and indeed we’ll start by looking at a section of the HTML file, Main.wo/Main.html.

 <p>The time is <webobject name=now_string></webobject>.</p>

This is clearly the paragraph that puts the date onto the screen, but you can see that there’s this weird <webobject> tag in there. That’s going to be substituted with our date object at runtime, but how? To answer this question, look at the Main.wo/Main.wod file.

now_string:WOString {value = now}

This file contains bindings, which if you’re familiar with Cocoa work pretty similar to Cocoa Bindings. This tells us that there’s a web object called now_string, that it’s an instance of WOString, and that its value binding is bound to something called now.

That now binding is actually a key on the component object. We might expect to see a method called -(NSString *)now defined on Main to fill in the value of this string. Indeed, if we look at Main.m:

- (NSString *)now {
  NSDate *theDate = [NSDate date];
  return [theDate description];

Where to go from here?

The start is to build more GSW apps, and of more complexity. However there are things that need to be done along the way: all of the rest of the code and files in this project are meaningless boilerplate. This stuff needs to be hidden, automated or removed. Then we can also automate the process: compare how much of a faff setting up this app was with running a Rails app in Heroku (and I haven’t even set up an adaptor to a real web server!). It’d be really useful to simplify common tasks in modern web applications, such as exposing REST interfaces and non-HTML document formats.


Objective-C is not just a client app language. It’s got a long heritage as a server technology, but unfortunately the tools for doing that are quite out of date. They’re still useful though, so people who have come into the Objective-C fold recently can expand the use of their newly-found skills. Hopefully people will adopt—and contribute to—GNUstepWeb and help everyone to make it a useful modern server technology.

posted by Graham at 23:32  

Sunday, March 25, 2012

More about the privacy pledge

Plenty of you have seen—and indeed signed— the App Makers’ Privacy Pledge on GitHub. If you haven’t, but after reading it are interested, see the instructions in the project README.

It’s great to see so many app makers taking an interest in this issue, and the main goal of the pledge is to raise awareness of app privacy concerns: awareness among developers that this is something to take seriously, and awareness among our customers that there are developers committed to respecting their identities and their data.

But awareness is useless if not followed through, so we need to do more. We need materials that developers can refer to: the GSM Association have good guidelines on app practices. We need actionable tasks that developers can implement right away, like Matt Gemmell’s hashing guide for social apps. We need sample code and libraries that developers can rely on. We need data lawyers to explain what the current regulations are, and what’s coming down the pipe. We need to convince the industry and the governments that we can regulate our own actions. We need the ability to audit our apps and determine whether they’re privacy-preserving. We need to be able to demonstrate to customers what we’ve done, and explain why that’s a good thing. We need to earn customer trust.

So there’s a lot to do, and the pledge is only the start. It’s off to a good start, but there’s still a long way to go.

posted by Graham at 21:06  

Friday, March 23, 2012

Confine ALL the things!

I was talking with Saul Mora at lunchtime about NSManagedObjectContext thread confinement. We launched into an interesting thought experiment: what if every object ran on its own thread?

This would be interesting. You can never use a method that returns a value, because then you’d need to block this object’s thread while waiting for that object. You’d need to rely solely on being able to pass messages from one object to another, without seeing return values. In other words, the “tell don’t ask” principle is enforced.

You get to avoid a lot of problems with concurrency issues in any app. Each object has its own data (something we’ve been able to get right for decades), and its own context to work on that data. No object should be able to trample on another object’s data because we have encapsulation, and that means that no thread should be able to trample on another thread’s data because one thread == one object.

Also notice that one object == one thread. You don’t have to worry about whether different methods on the same object are thread-safe, because you’re confining the object to a single thread so no code in that object will execute in parallel.

So yeah, I built that thing. Or really I built a very simple test app that demonstrates that thing in action. Check out the source from the GitHub project linked here. For your edification, here’s a description of the meat of the confinement behaviour, the FZAConfinementProxy. Starting with the interface:

@interface FZAConfinementProxy : NSProxy

- (id)initWithRemoteObject: (id)confinedObject;


This class is a NSProxy subclass, because all it needs to do is forward messages to another object. Or rather, that’s what it claims to do.

static const void *FZAProxyAssociation = (const void *)@"FZAConfinementProxyAssociationName";

@implementation FZAConfinementProxy {
    id remoteObject;
    NSOperationQueue *operationQueue;

- (id)initWithRemoteObject:(id)confinedObject {
    id existingProxy = objc_getAssociatedObject(confinedObject, FZAProxyAssociation);
    if (existingProxy) {
        self = nil;
        return existingProxy;

Ensure that there’s exactly one proxy per object, using Objective-C’s associated objects to map proxies onto implementations.

    remoteObject = confinedObject;
    if ([remoteObject isKindOfClass: [UIView class]]) {
        operationQueue = [NSOperationQueue mainQueue];
    } else {
        operationQueue = [[NSOperationQueue alloc] init];
        operationQueue.maxConcurrentOperationCount = 1;
    objc_setAssociatedObject(confinedObject, FZAProxyAssociation, self, OBJC_ASSOCIATION_ASSIGN);
    return self;

If we need to create a new proxy, then create an operation queue to which it can dispatch messages. UIKit needs to be used on the main thread, so ensure proxies for UIViews use the main operation queue.

What this means is that if your controller uses confinement proxies for both views and models (just as the dummy app does), then there’s no need to mess with methods like -performSelectorOnMainThread:withObject:waitUntilDone: in your controller logic. That’s automatically handled by the proxies.

- (NSMethodSignature *)methodSignatureForSelector:(SEL)sel {
    if (sel == @selector(initWithRemoteObject:)) {
        return [NSMethodSignature signatureWithObjCTypes: "@@:@"];
    else {
        return [remoteObject methodSignatureForSelector: sel];

Ensure that the Objective-C machinery can tell what messages it can send to this proxy.

- (void)forwardInvocation:(NSInvocation *)invocation {
    [invocation setTarget: remoteObject];
    NSInvocationOperation *operation = [[NSInvocationOperation alloc] initWithInvocation: invocation];
    [operationQueue addOperation: operation];

Here’s all the smarts, and it isn’t really very smart. Objective-C notices that the proxy object itself doesn’t implement the received method, and so asks the proxy whether it wants to forward the method. It’s, well, it’s a proxy, so it does want to. It rewrites the invocation target to point to the confined object, then adds the invocation to the operation queue.

That’s really it, bar some cleanup that you can see in GitHub. One thread per object limits the ways in which you can design code, so it’s a great way to learn about writing code within those limitations.

posted by Graham at 00:04  

Tuesday, March 13, 2012

How to TDD with CATCH

Plenty of people have asked me about the TDD framework I use. While the book Test-Driven iOS Development has code using OCUnit (for pragmatic, and previously-covered, reasons); I am currently more frequently to be discovered using Phil Nash’s CATCH framework. Here’s how you can get from New Project to first passing test. I’m starting in the same place that the book’s sample code starts: I need a Topic class that has a name.

Set up the project.

File->New Project in Xcode. Create an app project that matches your needs: an empty app will be fine. I’ll assume in this project, just as in the book, that you’ll be using Automatic Reference Counting for this project.

Now you need to add another app target. The “Empty Application” template is sufficient for this, and I call this target “Unit Tests”. When you’ve got that target set up, remove the app delegate class and the main.m file: we’ll add a test runner from CATCH.

For this target, you also have to choose whether or not to use ARC. If you enabled ARC for the app target, then the classes you’re testing will use Automatic Reference Counting, but the CATCH internals cannot because they use scary runtime hacks to dynamically generate classes from the test cases. In this walkthrough, you should enable ARC but we’ll have to edit compiler flags for a couple of files to get manual reference counting where we need it.

Check out CATCH from the GitHub project linked above. Then go into the Build Rules for your Unit Tests target, and add the path to the Catch/include source folder. For example, if you clone CATCH to the top-level folder of your Xcode project, you’ll need to add "$(SRCROOT)/Catch/include".

Locate the file Catch/projects/runners/iTchRunner/ (capitalisation as supplied) in Finder, and drag it onto your Xcode project. Add this file to the Unit Tests target. Now you need to go to the Build Phases inspector for your target, and under Compile Sources, next to add the flag -fno-objc-arc.

Add the first test

Add a new file to the Unit Tests target, called (you don’t need the matching header file). As with, you’ll need to set the -fno-objc-arc flag in the Build Phases. Include the catch.hpp header and write the test that checks for the name:

#include "catch.hpp"

TEST_CASE("Topic/Name", "Require that Topic has a name") {
    Topic *topic = [[Topic alloc] initWithName: @"iPhone"];
    REQUIRE(topic != nil);
    REQUIRE([ isEqualToString: @"iPhone"]);
    [topic release];

Get this test to compile

So currently, the test target won’t even build: we’re using a Topic class and its name property, and these don’t exist. Add a new file to the project, an NSObject class called Topic. You could add this file to both targets as it will be app code: don’t disable ARC here because the whole point is you want to write automatically reference-counted app code.

Just to fast-forward things a bit, here’s an implementation of Topic.[hm] that contains minimal implementations of the required methods.

#import <Foundation/Foundation.h>

@interface Topic : NSObject

@property (readonly) NSString *name;

- (id)initWithName: (NSString *)aName;


#import "Topic.h"

@implementation Topic

@synthesize name = _name;

- (id)initWithName:(NSString *)aName {
    if (self = [super init]) {
    return self;


Observe that the test fails.

Run the “Unit Tests” app in Xcode. You’ll see a screen like this:

Default Catch Screen

It’s not very exciting, but at least you can tell what to do: tap the “Run All Tests” button.

Catch failed test

Still not exciting, and it’s possible that if you have certain types of colour-blindness it’s not clear what’s happened. Luckily the logs are more explicit:

2012-03-13 17:18:01.640 UnitTests[960:707] Application windows are expected to have a root view controller at the end of application launch

2012-03-13 17:18:03.915 UnitTests[960:707] topic != __null succeeded for: 0x2d11b0 != __null

2012-03-13 17:18:03.917 UnitTests[960:707] [ isEqualToString: @"iPhone"] failed for: false

2012-03-13 17:18:03.923 UnitTests[960:707] 1 failures

Make this test pass

There’s only one line to add to the app code to get everything passing. Here’s the Topic implementation again with the change highlighted:

@implementation Topic

@synthesize name = _name;

- (id)initWithName:(NSString *)aName {
    if (self = [super init]) {
        _name = [aName copy];
    return self;


Run the tests again:

Passing CATCH tests

2012-03-13 17:46:17.644 UnitTests[1049:707] Application windows are expected to have a root view controller at the end of application launch

2012-03-13 17:46:43.746 UnitTests[1049:707] topic != __null succeeded for: 0x2ca7f0 != __null

2012-03-13 17:46:43.748 UnitTests[1049:707] [ isEqualToString: @"iPhone"] succeeded for: true

2012-03-13 17:46:43.753 UnitTests[1049:707] no failures



The runner for CATCH tests is currently very basic: it lets you know when things have passed or failed but not which tests have failed, though the tests are named. You need to manually prod it to get tests to run. However, the syntax is a refreshing change from the verbosity of JUnit-derived test frameworks.

posted by Graham at 17:18  

Monday, March 12, 2012

The security apprentice

This originally appeared in a post at Sophos’ Naked Security blog.

There have been two recent occasions on which my computing life has been influenced by Lord Sugar, the business mogul in charge of Amstrad and star of BBC One’s reality show The Apprentice.

The first was on a visit to the National Museum of Computing at Bletchley Park, where I got to use a computer Amstrad produced back in the 1980s, that went by the catchy name “CPC 6128”. It was running a Tetris clone called Blocks created—as was proudly proclaimed on the game screen—by an upstart programmer called “G Cluley”.

The second was this weekend, when the device you see in the picture showed up in a charity shop. This is the E-mailer Plus, a sort of executive phone/internet thing released by Amstrad in 2002. Being a fan of old computers, especially oddball ones like the E-mailer, I bought it.

The key feature of this phone was that it also had e-mail and web capabilities, albeit delivered via a premium rate number that lined Lord Sugar’s pockets with every mail check. Users could configure the phone to automatically fetch their mail to be read on the attached LCD screen.

And, indeed, someone had used this E-mailer for e-mail. Somone I shall call “Colin” had set up two accounts on the device. How do I know this? Because Colin hadn’t deleted these accounts before taking his phone to the charity shop.

As I said, the E-mailer relies on a dial-up service which was discontinued earlier this year by its ultimate owners, BSkyB. That means that I couldn’t, should I want to, fetch Colin’s new e-mail messages. But there were messages already stored on the phone that I could have read. More surprisingly, the configuration screens let me see passwords assigned to Colin’s accounts: has he used the same passwords on any other services?

Hopefully you’re aware of the need to ensure there’s no sensitive information stored on old computers before you dispose of them, particularly if you’re going to sell them on to other users. My new (or should I say Colin’s old) E-mailer shows that this goes for any device that stores or accesses your data, including phones both smart and retro.

I now imagine a scene in Lord Sugar’s office. “Colin, you made a basic error. By failing to delete your accounts before giving away your phone, you put your e-mail messages and your passwords at risk. You compromised the privacy of your own and your company’s data, and for that reason, you’re fired.”

posted by Graham at 16:03  

Thursday, March 1, 2012

On advertising’s place in the tech industry

Dave Winer said the way is open for a non-ad-supported tech sector:

The tech industry has been absorbed by the ad industry, and vice versa.
However, there is, imho, still room for a tech industry that is not merged with the ad industry.
In fact, if we want to have a tech industry at all, we’d better invest in the “other” one, because advertising isn’t much to bet on long-term. Seriously.

Daniel Jalkut went further, saying that ads are on the decline:

I’ll take this a step further: advertising is on the way out. Technology loathes a middle-man, and advertising as an industry is the king of all middle-men. The purpose of advertising is to connect customers with companies, so as to facilitate a transfer of money in exchange for goods or services. As time goes by, customers and companies will be more and more capable of achieving this on their own.

I agree with Dave, but not so much with Daniel. As I was writing this post, WPP (the world’s largest ad company) were on the radio announcing a 20% increase in profits worldwide, with an increase in the fraction of revenue they get from digital adverts up to 30%. WPP say that concerns of disintermediation (i.e. the removal of middle-men Daniel talks about) are unfounded. The internet has produced its own new middle men, particularly Google and Facebook. “Traditional” advertisers such as WPP are profiting from advertising with those companies.

Ads can be on the way out, but if that’s a world we want to see, it’s up to us app makers to do something about it, it won’t just happen automatically. Why us? Because the future involves even more mobile internet, and that means that we are in a privileged position to control what customers of the future are looking at. If we want them to look at something other than adverts, we need to put something that isn’t an advert and is worth looking at on their screen.

That’s a world in which many apps will need a very different business model than today’s. There are a lot—particularly on the web and on Android—of apps that are free to users and generate revenue from ads. Those would need to be funded in different ways: I don’t think that the freemium model of “pay to remove ads” will work. It doesn’t solve the goal of reducing ads (every user still sees the ad version at least once), for a start. Also, conversion rates will likely be low as people can “put up with” adverts.

Apps that still want to rely on viral growth still need a really low barrier to entry for new users, but need to change why and how many people convert to a paid version. Companies like Evernote and Dropbox have a model that appears to work for them, where paying customers get extra features/capacity over free users, and provide enough income to support the cost of providing the free product. And then of course there’s the shareware model that’s worked for decades: where the store you’re selling in allows you to do that.

But not all products need to be promoted virally. You can just sell something on the promise that it solves a problem, in the same way that I made a living selling solutions to app security problems, and the Omni Group make even more money selling the solution to the problem of remembering all the stuff you need to do.

You can also sell the promise that people can come back later: that is, you make money by letting customers stick to you. Look at iTunes Match. For some amount of money, Apple manages your music library. Then, next year, they’ll ask you whether you want to carry on doing that. You get the choice between pulling all your music back down and looking after it yourself again, or you can pay Apple once more and they’ll carry on doing it for a year. Apple are betting on there being enough people who come back every year to make this worthwhile.

So, to conclude, app makers who want to see an ad-free world are well placed to make that happen: doing so may start with an uncomfortable review of the way our apps are going to make money. In the short term, it’ll mean having an expensive fully-customer-funded app competing in a world of cheap ad-supported apps. But in the longer term it could result in better, more transparent relationships with our customers who aren’t worried about what data we’re selling to advertisers.

posted by Graham at 09:37  

Powered by WordPress