Intuitive is the Enemy of Good

In the previous instalment, I discussed an interview in which Alan Kay maligned growth-restricted user interfaces. Here’s the quote again:

There is the desire of a consumer society to have no learning curves. This tends to result in very dumbed-down products that are easy to get started on, but are generally worthless and/or debilitating. We can contrast this with technologies that do have learning curves, but pay off well and allow users to become experts (for example, musical instruments, writing, bicycles, etc. and to a lesser extent automobiles).

This is nowhere more evident than in the world of the mobile app. Any one app comprises a very small number of very focussed, very easy to use features. This has a couple of different effects. One is that my phone as a whole is an incredibly broad, incredibly shallow experience. For example, one goal I want help with from technology is:

As an obese programmer, I want to understand how I can improve my lifestyle in order to live longer and be healthier.

Is there an app for that? No; I have six apps that kindof together provide an OK, but pretty disjointed experience that gets me some dissatisfying way toward my goal. I can tell three of these apps how much I run, but I have to remember that some subset can feed information to the others but the remainder cannot. I can tell a couple of them how much I ate, but if I do it in one of them then another won’t count it correctly. Putting enough software to fulfil my goal into one app presumably breaks the cardinal rule of making every feature available within two gestures of the app’s launch screen. Therefore every feature is instead hidden behind the externalised myriad gestures required to navigate my home screens and their folders to get to the disparate subsets of utility.

The second observable effect is that there is a lot of wasted potential in both the device, and the person operating that device. You have never met an expert iPhone user, for the simple reason that someone who’s been using an iPhone for six years is no more capable than someone who has spent a week with their new device diligently investigating. There is no continued novelty, there are no undiscovered experiences. There is no expertise. Welcome to the land of the perpetual beginner.

Thankfully, marketing provided us with a thought-terminating cliché, to help us in our discomfort with this situation. They gave us the cars and trucks analogy. Don’t worry that you can’t do everything you’d expect with this device. You shouldn’t expect to do absolutely everything with this device. Notice the sleight of brain?

Let us pause for a paragraph to notice that even if making the most simple, dumbed-down (wait, sorry, intuitive) experience were our goal, we use techniques that keep that from within our grasp. An A/B test will tell you whether this version is incrementally “better” than that version, but will not tell you whether the peak you are approaching is the tallest mountain in the range. Just as with evolution, valley crossing is hard without a monumental shake-up or an interminable period of neutral drift.

Desktop environments didn’t usually get this any better. The learning path for most WIMP interfaces can be listed thus:

  1. cannot use mouse.
  2. can use mouse, cannot remember command locations.
  3. can remember command locations.
  4. can remember keyboard shortcuts.
  5. ???
  6. programming.

A near-perfect example of this would be emacs. You start off with a straightforward modeless editor window, but you don’t know how to save, quit, load a file, or anything. So you find yourself some cheat-sheet, and pretty soon you know those things, and start to find other things like swapping buffers, opening multiple windows, and navigating around a buffer. Then you want to compose a couple of commands, and suddenly you need to learn LISP. Many people will cap out at level 4, or even somewhere between 3 and 4 (which is where I am with most IDEs unless I use them day-in, day-out for months).

The lost magic is in level 5. Tools that do a good job of enabling improvement without requiring that you adopt a skill you don’t identify with (i.e. programming, learning the innards of a computer) invite greater investment over time, rewarding you with greater results. Photoshop gets this right. Automator gets it right. AppleScript gets it wrong; that’s just programming (in fact it’s all the hard bits from Smalltalk with none of the easy or welcoming bits). Yahoo! Pipes gets it right but markets it wrong. Quartz Composer nearly gets it right. Excel is, well, a bit of a boundary case.

The really sneaky bit is that level 5 is programming, just with none of the trappings associated with the legacy way of programming that professionals do it. No code (usually), no expressing your complex graphical problem as text, no expectation that you understand git, no philosophical wrangling over whether squares are rectangles or not. It’s programming, but with a closer affinity with the problem domain than bashing out semicolons and braces. Level 5 is where we can enable people to get the most out of their computers, without making them think that they’re computering.

Standing at the Crossroads

A while back I wrote Conflicts in my Mental Model of Objective-C, in which I listed a few small scale dichotomies or cognitive dissonances that plagued my notion of my work. I just worked out what the overall picture is, the jigsaw into which all of these pieces can be assembled.

And I do mean just. It’s about 1AM on Christmas Eve, but this picture hit me so hard I couldn’t stop thinking about it without writing it down and getting it out of my head. If it doesn’t explain everything, it shows me the shape of the solution at least.

A tale of two Apples

I believe that everything I wrote in the Conflicts post can be understood in terms of two different and (of course) opposed models of Apple. I also believe that the two models are irreconcilable, but that the opposition is also accidental, not essential. That by removing the supposed conflict between them, everything I thought was a problem can be resolved.

It was the best of iPads

The iPad is, I would argue (and accept that this is a subjective argument) the most tasteful application of computing technology to the world of many computer users. I would further argue, and this is perhaps on firmer ground, that the reason the iPad is so tasteful is because Apple spend a lot more time and resources on worrying about questions of taste than many of their competitors and others in the industry.

There are many visions that have combined to produce the iPad, but interestingly the one that I think is clearest is John Scully’s Knowledge Navigator. In the concept videos for Knowledge Navigator, a tablet computer with natural language speech comprehension and a multi-touch screen is able to use the many hyperlinked documents available on the Web to answer a wide range of questions, make information available to its users and even help them to plan their schedules.

This is what we have now. This is the iPad, with Siri and a host of third-party applications. Apple even used to use the slogan “there’s an app for that”. Do you have a problem? You can probably solve it with iOS and a trip to the app store.

It was the worst of iPads

OK, what do you do if your problem isn’t solved on the app store, or the available solutions aren’t satisfactory?

Well first, you’d better get yourself another computer because while the iPad is generally designed for solving problems it isn’t designed for solving general problems. You might be able to find some code editors on the iPad, but you sure aren’t going to use them to write an iPad app without external assistance.

OK, so you’ve got your computer, and you’ve learned how to do the stuff that makes iPad apps. Now you just pay a recurring fee to be allowed to put that stuff onto your iPad. And what if you want to share that with your friends? Only if it meets Apple’s approval.

If the iPad is the Knowledge Navigator, it is not the Dynabook. A Dynabook is a computer that you can use to solve your problems on, but it’s also one on which you can create solutions to your own problems.

The promise of the Dynabook is that if you understand what your problem is, you can model that problem on the Dynabook. You model it with objects-either your own or ones supplied for you. You can change and create these objects until they model the problem you have, at which point you can use them to compute a solution.

The irony is that we have all the parts needed in a Dynabook, all in the iPad. Computer so simple even children can use it? Check. Objects? Check. Repositories of objects created by other people so we don’t have to rewrite our own basic objects all the time? We call that CocoaPods (or RubyGems, or whatever the poison in your area of the world). But we just can’t put all of these things together on that computer itself.

That would be distasteful. That might let people do things that make the iPad look bad. That might mean iPads providing experiences that haven’t been vetted by the mothership.

Does using an iPad ever make you wonder how iPads work? What they can do? What you can make them do? You can answer these questions, but not using an iPad. Your Knowledge Navigator does not know the route to that particular destination.

This is what is truly meant when it is said that the iPad is not upgradeable. Forget swapping out memory chips or radio transmitters. Those are just lumps of sand inside a box made of melted sand and refined rock. The iPad is not upgradeable because you are stuck with the default experience: the out-of-the-box facilities plus those that have been approved from on high. It might be good, but it might not be good enough.

Notice that this is not an “everyone must program” position. That would be a very bad experience. The position is rather “everyone must have the facility, should they be so inclined, to make their computer better for them than the manufacturers did”.

Conclusions?

I think that the Apple described above is not at the intersection of technology and the liberal arts. It is at the border, a self-appointed barrier of things that might flow between the two.

I believe that the two visions can be reconciled, and that a thing can be both the Knowledge Navigator and the Dynabook. I don’t believe you have to disable some experiences to provide others. I believe that Apple the champions of tasteful computing can be applauded at every turn while Apple the high priests of the church of computing can be fought tooth and nail.

Enablers? Yes, please. Arbiters? No, thanks.

Garbage-collected Objective-C

When was a garbage collector added to Objective-C? If you follow Apple’s work with the language, you might be inclined to believe that it was in 2008 when AutoZone was added as part of Objective-C 2.0 (the AutoZone collector has since been deprecated by Apple, and I’m not sure whether anyone else ever adopted it).

With a slightly wider knowledge of the language’s history, you can push this date back a bit. The GNUstep project—a Free Software reimplementation of Apple’s (formerly NeXT’s) APIs—has been using the Boehm–Demers–Weiser collector for a while. How long? I can’t tell exactly, but a keyword search in the project’s version control logs makes me think that most of the work to support it was done by one person in mid-2002:

r13976 | nico | 2002-06-26 15:34:16 +0100 (Wed, 26 Jun 2002) | 3 lines

Do not add -lobjc_gc -lgc flags when compiling with gc=yes – should now
be added automatically by gnustep-make


r13971 | nico | 2002-06-25 18:28:56 +0100 (Tue, 25 Jun 2002) | 2 lines

Tidyup for gc=yes with old compilers


r13970 | nico | 2002-06-25 18:23:05 +0100 (Tue, 25 Jun 2002) | 2 lines

Tidied code to compile with gc=yes and older compilers


r13969 | nico | 2002-06-25 13:36:11 +0100 (Tue, 25 Jun 2002) | 3 lines

Tidied some indentation; a couple of insignificant changes to have it compile
under gc


r13968 | nico | 2002-06-25 13:15:04 +0100 (Tue, 25 Jun 2002) | 2 lines

Tidied code which wouldn’t compile with gc=yes and gcc < 3.x


r13967 | nico | 2002-06-25 13:13:19 +0100 (Tue, 25 Jun 2002) | 2 lines

Tidied code which was not compiling with the garbage collector


r13966 | nico | 2002-06-25 13:12:17 +0100 (Tue, 25 Jun 2002) | 2 lines

Tidied code which was not compiling with gc=yes

That was, until fairly recently, the earliest example I knew about. Then I discovered a conference talk by Paulo Ferreira:

Reclaiming storage in an object oriented platform supporting extended C++ and Objective-C applications

This is a paper presented at “1991 International Workshop on Object Orientation in Operating Systems”. 1991. That is—obviously—11 years before GNUstep’s GC work and 17 years before Apple released AutoZone.

Comandos

The context in which this work was being done is a platform called Comandos. I’d never heard of that before—and I thought I knew Objective-C!

Judging from the report linked above, Comandos is a platform for distributed and parallel object-oriented software, based on UNIX but supporting multiple variants. The fact that it was created in 1986 means that both the languages supported—Objective-C and C++—were new at the time. Indeed the project was contemporary with the development of NeXTSTEP, which was publicly released to developers in 1988.

The 1994 summary report doesn’t mention Objective-C: just C++, Eiffel and a bespoke language called Guide. It’s possible that the platform supported ObjC simply because they used gcc which picked up ObjC support during the life of Comandos; however this seems unlikely as there would be significant work in making Objective-C objects work with their platform’s distributed messaging interface and persistence subsystem.

Why ObjC should be one of two languages mentioned (along with C++) in the 1991 paper on garbage collection, but zero of three mentioned (C++, Eiffel, Guide) in 1994 will have to remain a mystery for now. Looking into the references for Ferreira’s paper, I can find one mention of Objective-C as the inspiration for their own, custom C-based message dispatch system, but no indication that they actually used Objective-C.

The Garbage Collector

I’m not really an expert at garbage collectors. In fact, I have no idea what I’m doing. I appreciate them when they’re around, and leak or crash things occasionally when they’re not.

To my uneducated eye, the description of the Ferreira 1991 garbage collector and Apple’s description of their collector (no link I’m afraid, it was session 940 at WWDC 2008) look quite different. AutoZone is conservative (like B-W-D) and only works on Objective-C objects. Ferreira’s collector operates, like B-W-D, on any memory block including new C++ instances and C heap allocations. Apple’s collector is supposed to avoid blocking wherever it can, a constraint not mentioned in the Ferreira paper.

All of Comandos, GNUstep and Cocoa (Apple’s Objective-C framework) have systems for distributed objects that complicate collection: does some remote process have a handle on memory in my address space? The proxy system used by Cocoa and GNUstep make it easy to answer this question. Comandos used a different technique, where objects local to a process were “volatile” and objects shared between processes were “persistent”. Persistent objects were subject to a different lifecycle management process, so the Ferreira GC didn’t interact with them.

As an aside, Apple’s garbage collector also needed to provide a “mixed mode”—support for code that could be loaded into either a garbage-collected or manually managed process.

Conclusions

Memory management is hard. Making programmers do it themselves leads to all sorts of problems. Doing it automatically is also hard, and many different approaches have been tried over the last few decades. Interestingly, Apple has (for the moment) settled on a “none of the above” approach, using a compiler-inserted reference counting system based on the manual ownership tracking previously implemented by the frameworks.

What interests me most about this paper on Objective-C garbage collection is not so much its technical content (which it’s actually rather light on, containing only conversational overviews of the algorithms and no information about results), but the fact that it existed at all and I, as someone who considers himself an experienced Objective-C programmer, did not know anything about it or its project.

That’s why I started this blog [Ed: referring to the blog these posts are imported from] by discussing it. A necessary prerequisite to deciding whether the literature has something useful to tell us is knowing about its existence. I’m really surprised that it took so long for me to find out about something that’s almost directly related to my everyday work. Mind you, maybe I shouldn’t feel too bad: the author of AutoZone told me he hadn’t heard of it, either.

Lighter UIViewControllers

The first issue of Objective-C periodical objc.io has just been announced:

Issue #1 is about lighter view controllers. The introduction tells you a bit more about this issue and us. First, Chris writes about lighter view controllers. Florian expands on this topic with clean table view code. Then Daniel explores view controller testing. Finally, in our first guest article, Ricki explains view controller containment.

Each of the articles is of a high quality, I thoroughly recommend that you read this. I powered through it this morning and have already put myself on the mailing list for the next issue. I think it’s worth quite a few dollars more than the $0 they’re asking.

On a more self-focussed note, I’m pleased to note that two of the articles cite my work. This isn’t out of some arrogant pleasure at seeing my name on someone else’s website. One of my personal goals has been to teach the people who teach the other people: to ensure that what I learn becomes a part of the memeplex and isn’t forgotten and reinvented a few years later. In APPropriate Behaviour a few of the chapters discuss the learning and teaching of software making. I consider it one of the great responsibilities of our discipline, to ensure the mistakes we had to make are not made by those who come after us.

The obj.io team have done a great job of researching their articles, taking the knowledge that has been found and stories that have been told and synthesising new knowledge and new stories from them. I’m both proud and humble about my small, indirect role in this effort.

A two-dimensional dictionary

What?

A thing I made has just been open-sourced by my employers at Agant: the AGTTwoDimensionalDictionary works a bit like a normal dictionary, except that the keys are CGPoints meaning we can find all the objects within a given rectangle.

Why?

A lot of time on developing Discworld: The Ankh-Morpork Map was spent on performance optimisation: there’s a lot of stuff to get moving around a whole city. As described by Dave Addey, the buildings on the map were traced and rendered into separate images so that we could let characters go behind them. This means that there are a few thousand of those little images, and whenever you’re panning the map around the app has to decide which images are visible, put them in the correct place (in three dimensions; remember people can be in front of or behind the buildings) and draw everything.

A first pass involved creating a set containing all of the objects, looping over them to find out which were within the screen region. This was too slow. Implementing this 2-d index instead made it take about 20% the original time for only a few tens of kilobytes more memory, so that’s where we are now. It’s also why the data type doesn’t currently do any rebalancing of its tree; it had become fast enough for the app it was built in already. This is a key part of performance work: know which battles are worth fighting. About one month of full-time development went into optimising this app, and it would’ve been more if we hadn’t been measuring where the most benefit could be gained. By the time we started releasing betas, every code change was measured in Instruments before being accepted.

Anyway, we’ve open-sourced it so it can be fast enough for your app, too.

How?

There’s a data structure called the multidimensional binary tree or k-d tree, and this dictionary is backed by that data structure. I couldn’t find an implementation of that structure I could use in an iOS app, so cracked open the Objective-C++ and built this one.

Objective-C++? Yes. There are two reasons for using C++ in this context: one is that the structure actually does get accessed often enough in the Discworld app that dynamic dispatch all the way down adds a significant time penalty. The other is that the structure contains enough objects that having a spare isa pointer per node adds a significant memory penalty.

But then there’s also a good reason for using Objective-C: it’s an Objective-C app. My controller objects shouldn’t have to be written in a different language just to use some data structure. Therefore I reach for the only application of ObjC++ that should even be permitted to compile: an implementation in one language that exposes an interface in the other. Even the unit tests are written in straight Objective-C, because that’s how the class is supposed to be used.

Sideloading content into iOS apps

All non-trivial apps visualise content in some form, whether it’s game levels embedded in the app, data loaded from some internet service, or something else.

In many cases the developer who’s writing the Objective-C code isn’t going to be the person who creates or prepares this content. In the case of embedded content, this can lead to a slow feedback loop—the content experts create a database or some other assets, then send it to the developer. The developer prepares a build using the new assets, uploading it to TestFlight or some other ad-hoc distribution centre. Then the content people can download that app to see their content in the context of the application it’s designed for.

There’s a simple way to close this loop, letting content creators see the app with their latest changes as they make them. That is to use iTunes File Sharing to load the content via the app’s Documents folder.

If you have a line like this:

NSString *pathToContent = [[NSBundle mainBundle] pathToResource: @"myDatabase" ofType: @"sqlite"];

Change it to use a function like this:

NSString *pathToPotentiallySideloadedFile(NSString *filename, NSString *type)
{
    NSString *pathInDocumentsFolder = [[[NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) lastObject] stringByAppendingPathComponent: filename] stringByAppendingPathExtension: type];
    if (pathInDocumentsFolder)
        return pathInDocumentsFolder;
    else
        return [[NSBundle mainBundle] pathForResource: filename ofType: type];
}

//...
NSString *pathToContent = pathToPotentiallySideloadedFile(@"myDatabase", @"sqlite");

Now if people working on your app have a file in their Documents folder with the same name as the one used in the app, it’ll load their version. So, how do they get it in there?

You need to make a simple change to your app’s Info.plist:

    <key>UIFileSharingEnabled</key>
    <true/>

Now when anybody with the app connects their device to iTunes, they’ll be able to use file sharing to add their own content. Don’t forget to turn this off before you go live!

I mentioned at the beginning of this post that this technique can be used for networked apps. Obviously there isn’t really any difficulty getting updated content into a network-driven app; or if there is, someone did it wrong.

It’s the opposite problem you have: keeping the content fixed. If your online component—be it a CMS, a data feed from an API, or something else—is getting new data you can’t always ensure that the app is looking at the same stuff in testing. Indeed, sometimes I’ve found the CMS developers changing the data format without telling anyone; if you’re investigating a particular condition related to the state of the data, it can be hard to reproduce.

You can use the iTunes File Sharing technique to load a specific version of the app’s data without relying on the network connection and the server giving you the same output. This is great for regression testing, as you can ensure that only your code is changing between runs.

The debugger of royalty

We’ve all got little libraries of code or scripts that help us with debugging. Often these are for logging information in a particular way, or wrapping logs/tests such that they’re only invoked in Debug builds but not in production. Or they clean up your IDE’s brainfarts.

Having created these debug libraries, how are you going to get your production code to use them? Are you really going to sprinkle MyCompanyDebugLog(fmt,…) messages throughout your app?

Introducing step one on the road to sanity: the debug proxy. This is useful when you want to find out how a particular class gets used, e.g. when it provides callbacks that will be invoked by a framework. You can intercept all the messages to the object, and inspect them as you see fit. Here’s the code (written with the assumption that ARC is enabled):

FZADebugProxy.h

#import <Foundation/Foundation.h>

@interface FZADebugProxy : NSProxy

- (id)initWithTarget: (NSObject *)aTarget;

@end

FZADebugProxy.m

#import "FZADebugProxy.h"

@implementation FZADebugProxy {
    NSObject *target;
}

- (id)initWithTarget:(NSObject *)aTarget {
    target = aTarget;
    return self;
}

- (NSMethodSignature *)methodSignatureForSelector:(SEL)sel {
    NSMethodSignature *signature = [target methodSignatureForSelector: sel];
    if (signature == nil) {
        signature = [super methodSignatureForSelector: sel];
    }
    return signature;
}

- (BOOL)respondsToSelector:(SEL)aSelector {
    return [target respondsToSelector: aSelector] ? YES : [super respondsToSelector: aSelector];
}

- (void)forwardInvocation:(NSInvocation *)invocation {
    invocation.target = target;
    SEL aSelector = [invocation selector];
    (void)aSelector;
    [invocation invoke];
}

@end

And no, there isn’t a bug in the -initWithTarget: method. The slightly clumsy extraction of the selector in -forwardInvocation: is done to avoid a common problem with using Objective-C inside the debugger where it decides it doesn’t know the return type of objc_msgSend() and refuses to call the method.

You would use it like this. Here, I’ve modified the app delegate from BrowseOverflow to use a proxy object for the object configuration – a sort of domain-specific IoC container.

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
    BrowseOverflowViewController *firstViewController = [[BrowseOverflowViewController alloc] initWithNibName: nil bundle: nil];
    firstViewController.objectConfiguration = (BrowseOverflowObjectConfiguration *)[[FZADebugProxy alloc] initWithTarget: [[BrowseOverflowObjectConfiguration alloc] init]];
    TopicTableDataSource *dataSource = [[TopicTableDataSource alloc] init];
    [dataSource setTopics: [self topics]];
    firstViewController.dataSource = dataSource;
    self.navigationController.viewControllers = [NSArray arrayWithObject: firstViewController];
    self.window.rootViewController = self.navigationController;
    [self.window makeKeyAndVisible];
    return YES;
}

The bold line is the important change. The cast silences the compiler’s strict type-checking when it comes to property assignment, because it doesn’t believe that NSProxy is of the correct type. Remember this is only debug code that you’re not going to commit: you could switch to a plain old setter, suppress the warning using diagnostic pragmas or do whatever you want here.

At this point, it’s worth running the unit tests and using the app to convince yourself that the behaviour hasn’t changed at all.

So, how do you use it? Shouldn’t there be an NSLog() or something in the proxy class so you can see when the target’s messaged?

No.

Step two on the road to sanity is to avoid printf()-based debugging in all of its forms. What you want to do here is to use Xcode’s debugger actions so that you don’t hard-code your debugging inspection capabilities into your source code.

Set a breakpoint in -[FZADebugProxy forwardInvocation:]. This breakpoint will be met whenever the target object is messaged. Now right-click on the breakpoint marker in the Xcode source editor’s gutter and choose “Edit Breakpoint…” to bring up this popover.

Xcode's breakpoint popover.

In this case, I’ve set the breakpoint to log the selector that was invoked, and crucially to continue after evaluation so that my app doesn’t stop in the debugger every time the target object is messaged. After a bit of a play with the app in the simulator, the debug log looks like this:

GNU gdb 6.3.50-20050815 (Apple version gdb-1752) (Sat Jan 28 03:02:46 UTC 2012)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin".sharedlibrary apply-load-rules all
Attaching to process 3898.
Pending breakpoint 1 - ""FZADebugProxy.m":37" resolved
Current language:  auto; currently objective-c
0x10271:	 "stackOverflowManager"
0x102a0:	 "avatarStore"
0x10271:	 "stackOverflowManager"
0x10271:	 "stackOverflowManager"
0x102a0:	 "avatarStore"

Pretty nifty, yes? You can do a lot with Xcode’s breakpoint actions: running a shell script or an AppleScript are interesting options (you could have Xcode send you an iMessage every time it send your target an Objective-C message). Speaking out the names of selectors is fun for a very short while, but not overly useful.

Xcode’s breakpoint actions give you a much more powerful debugging capability than NSLog(). By using breakpoint actions on an Objective-C proxy object, you can create highly customisable aspect-oriented techniques for debugging your code.

Why we don’t trust -retainCount

I’m pretty sure @bbum must have worn through a few keyboards telling users of StackOverflow not to rely on the value of an Objective-C object’s -retainCount. Why? When we create an object, it has a retain count of 1, right? Retains (and, for immutable objects, copies) bump that up, releases (and, some time later, autoreleases) bring it down, right? If an attempt to release would bring the retain count to 0, that object gets released, right? Right?!?

Well, that’s not true for all objects, but leaving that aside, your code isn’t the only code running in your process. The system libraries along with any third-party code you’ve included in your app are all doing things, including retaining and releasing objects. Let’s take a look at a specific case of that.

The code in this post is from Gnustep-base, the LGPL implementation of OpenStep’s Foundation library. The behaviour shown is identical to behaviour in Apple’s Foundation. Here’s (most of) +[NSNumber initialize].

/*
 * Numbers from -1 to 12 inclusive that are reused.
 */
static NSNumber *ReusedInstances[14];
static NSBoolNumber *boolY;		// Boolean YES (integer 1)
static NSBoolNumber *boolN;		// Boolean NO (integer 0)

+ (void) initialize
{
  int i;

  if ([NSNumber class] != self)
    {
      return;
    }

  // ...

  boolY = NSAllocateObject (NSBoolNumberClass, 0, 0);
  boolY->value = 1;
  boolN = NSAllocateObject (NSBoolNumberClass, 0, 0);
  boolN->value = 0;

  for (i = 0; i < 14; i++)
    {
      NSIntNumber *n = NSAllocateObject (NSIntNumberClass, 0, 0);

      n->value = i - 1;
      ReusedInstances[i] = n;
    }
}

We see that sixteen instances of NSNumber subclasses, representing YES, NO and the integers -1 to 12, have been allocated and stored away in static variables. What’s that about? Well here’s (most of) +numberWithInt:.

/*
 * Macro for checking whether this value is the same as one of the singleton
 * instances.  
 */
#define CHECK_SINGLETON(aValue) \
if (aValue >= -1 && aValue <= 12)\
{\
  return ReusedInstances[aValue+1];\
}

// ...

+ (NSNumber *) numberWithInt: (int)aValue
{
  NSIntNumber *n;

  if (self != NSNumberClass)
    {
      return [[[self alloc] initWithBytes: (const void *)&aValue
        objCType: @encode(int)] autorelease];
    }

  CHECK_SINGLETON (aValue);
  n = NSAllocateObject (NSIntNumberClass, 0, 0);
  n->value = aValue;
  return AUTORELEASE(n);
}

So it looks like the retain count of an object you get back from +numberWithInt: depends on the value you pass, and whether anyone else is trying to use a number with the same value right now.

In other words, while it’s easy to get an object’s retain count using the -retainCount method, the number you expect to see may well be wrong. The real value depends on so many parameters that it’s reasonable to conclude that you don’t know what it’s expected to be right now, so don’t depend on it.

By the way, the NSNumber implementation is a great place to find out more about how Foundation is built. It’s a class cluster (which is why you can see NSIntNumber and NSBoolNumber classes), and you can see how to use marker pointers to encode the value of an NSNumber right into the object pointer.

On explaining stuff to people

An article that recently made the rounds, though it was written back in September, is called Apple’s Idioten Vektor. It’s a discussion of how the CCCrypt() function in Apple’s CommonCrypto library, when used in its default cipher block chaining mode, treats the IV (Initialization Vector) parameter as optional. If you don’t supply an IV, it provides its own IV of 0x0.

Professional Cocoa Application Security also covers CommonCrypto, CBC mode, and the Initialization Vector. Pages 79-88 discuss block encryption. The section includes sample code for both one-shot and staged use of the API. It explains how to set the IV using a random number generator, and why this should be done.[1] Mercifully when the author of the above blog post reviewed the code in my book section, he decided I was doing it correctly.

So both publications cover the same content. There’s a clear difference in presentation technique, though. I realise that the blog post is categorised as a “rant” by the author, and that I’m about to be the pot that calls the kettle black. However, I do not believe that the attitude taken in the post—I won’t describe it, you can read it—is constructive. Calling people out is not cool, helping them get things correct is. Laughing at the “fail” is not something that endears people to us, and let’s face it, security people could definitely be more endearing. We have a difficult challenge: we ask developers to do more work to bring their products to market, to spend more money on engineering (and often consultants), in return for potentially protecting some unquantified future lost revenue and customer hardship.

Yes there is a large technical component in doing that stuff, but solving the above challenge also depends very strongly on relationship management. Security experts need to demonstrate that we’re all on the same side; that we want to work with the rest of the software industry to help make better software. Again, a challenge arises: a lot of the help provided by security engineers comes in the form of pointing out mistakes. But we shouldn’t be self promoting douchebags about it. Perhaps we’re going about it wrong. I always strive to help the developers I work with by identifying and discussing the potential mistakes before they happen. Then there’s less friction: “we’re going to do this right” is a much more palatable story than “you did this wrong”.

On the other hand, the Idioten Vektor approach generated a load of discussion and coverage, while only a couple of thousand people ever read Professional Cocoa Application Security. So there’s clearly something in the sensationalist approach too. Perhaps it’s me that doesn’t get it.

[1]Note that the book was written while iPhone OS 3 was the current version, which is why the file protection options are not discussed. If I were covering the same topic today I would recommend eschewing CCCrypto for all but the most specialised of purposes, and would suggest setting an appropriate file protection level instead. The book also didn’t put encryption into the broader context of cryptographic protocols; a mistake I have since rectified.

On SSL Pinning for Cocoa [Touch]

Moxie Marlinspike, recently-acquired security boffin at Twitter, blogged about SSL pinning. The summary is that relying on the CA trust model to validate SSL certificates introduces some risk into using an app – there are hundreds of trusted roots in an operating system like iOS, and you don’t necessarily want to trust all (or even any) of the keyholders. Where you’re connecting to a specific server under your control, you don’t need anyone else to tell you the server’s identity: you know what server you need to use, you should just look for its certificate. Then it doesn’t matter if someone compromises any CA; you’re not trusting the CAs any more. He calls this SSL pinning, and it’s something I’ve recommended to Fuzzy Aliens clients over the past year. I thought it’d be good to dig into how you do SSL pinning on Mac OS X and iOS.

The first thing you need to do is to tell Foundation not to evaluate the server certificate itself, but to pass the certificate to you for checking. You do this by telling the NSURLConnection that its delegate can authenticate in the “server trust” protection space.

-(BOOL)connection:(NSURLConnection *)connection canAuthenticateAgainstProtectionSpace:(NSURLProtectionSpace *)space {
  return [[space authenticationMethod] isEqualToString: NSURLAuthenticationMethodServerTrust];
}

Now your NSURLConnection delegate will receive an authentication challenge when the SSL connection is negotiated. In this authentication challenge, you evaluate the server trust to discover the certificate chain, then look for your certificate on the chain. Because you know exactly what certificate you’re looking for, you can do a bytewise comparison and don’t need to do anything like checking the common name or extracting the fingerprint: it either is your certificate or it isn’t. In the case below, I look only at the leaf certificate, and I assume that the app has a copy of the server’s cert in the sealed app bundle at MyApp.app/Contents/Resources/servername.example.com.cer.


- (void)connection:(NSURLConnection *)connection didReceiveAuthenticationChallenge:(NSURLAuthenticationChallenge *)challenge {
  if ([[[challenge protectionSpace] authenticationMethod] isEqualToString: NSURLAuthenticationMethodServerTrust]) {
    SecTrustRef serverTrust = [[challenge protectionSpace] serverTrust];
    (void) SecTrustEvaluate(serverTrust, NULL);
    NSData *localCertificateData = [NSData dataWithContentsOfFile: [[NSBundle mainBundle]
                                                                    pathForResource: serverName
                                                                    ofType: @"cer"]];
    SecCertificateRef remoteVersionOfServerCertificate = SecTrustGetCertificateAtIndex(serverTrust, 0);
    CFDataRef remoteCertificateData = SecCertificateCopyData(remoteVersionOfServerCertificate);
    BOOL certificatesAreTheSame = [localCertificateData isEqualToData: (__bridge NSData *)remoteCertificateData];
    CFRelease(remoteCertificateData);
    if (certificatesAreTheSame) {
      [[challenge sender] useCredential: [NSURLCredential credentialForTrust: serverTrust] forAuthenticationChallenge: challenge];
    }
    else {
      [[challenge sender] cancelAuthenticationChallenge: challenge];
    }
  }
  // fall through for challenges in other spaces - or respond to them if you need to
}

That’s really all there is to it. You may want to change some of the details depending on your organisation: for example you may issue your own intermediate or root certificate and check for its presence while allowing the leaf certificate to vary; however the point is to get away from the SSL certificate authority trust model so I haven’t shown that here.