APPropriate Behaviour is complete!

APPropriate Behaviour, the book on things programmers do that aren’t programming, is now complete! The final chapter – a philosophy of software making – has been added, concluding the book.

Just because it’s complete, doesn’t mean it’s finished: as my understanding of what we do develops I’ll probably want to correct things, or add new anecdotes or ideas. Readers of the book automatically get free updates whenever I create them in the future, so I hope that this is a book that grows with us.

As ever, the introduction to the book has instructions on joining the book’s Glassboard to discuss the content or omissions from the content. I look forward to reading what you have to say about the book in the Glassboard.

While the recommended purchase price of APPropriate Behaviour is $20, the minimum price now that it’s complete is just $10. Looking at the prices paid by the 107 readers who bought it while it was still being written, $10 is below the median price (so most people chose to pay more than $10) and the modal price (so the most common price chosen by readers was higher than $10).

A little about writing the book: I had created the outline of the book last Summer, while thinking about the things I believed should’ve been mentioned in Code Complete but were missing. I finally decided that it actually deserved to be written toward the end of the year, and used National Novel Writing Month as an excuse to start on the draft. A sizeable portion of the draft typescript was created in that month; enough to upload to LeanPub and start getting feedback on from early readers. I really appreciate the help and input those early readers, along with other people I’ve talked to the material about, have given both in preparing APPropriate Behaviour and in understanding my career and our industry.

Over the next few months, I tidied up that first draft, added new chapters, and extended the existing material. The end result – the 11th release including that first draft – is 141 pages of reflection over the decade in which I’ve been paid to make software: not a long time, but still nearly 15% of the sector’s total lifespan. I invite you to grab a copy from LeanPub and share in my reflections on that decade, and consider what should happen in the next.

Posted in advancement of the self, books, Responsibility, social-science, software-engineering | Leave a comment

When single responsibility isn’t possible

This posted was motivated by Rob Rix’s bug report on NSObject, “Split NSObject protocol into logical sub-protocols”. He notes that NSObject provides multiple responsibilities[*]: hashing, equality checking, sending messages, introspecting and so on.

What that bug report didn’t look at was the rest of NSObject‘s functionality that isn’t in the NSObject protocol. The class itself defines method signature lookups, message forwarding and archiving features. Yet more features are added via categories: scripting support (Mac only), Key-Value Coding and Key-Value Observing are all added in this way.

I wondered whether this many responsibilities in the root class were common, and decided to look at other object libraries. Pretty much all Objective-C object libraries work this way: the Object class from ObjPak, NeXTSTEP and ICPak101 (no link, sadly) all have similarly rambling collections of functionality.

[*] By extension, all subclasses of NSObject and NSProxy (which _also_ conforms to the NSObject protocol) do, too.

Another environment I’ve worked a lot in is Java. The interface for java.lang.Object is mercifully brief: it borrows NSObject‘s ridiculous implementation of a copy method that doesn’t work by default. It actually has most of the same responsibilities, though notably not introspection nor message-sending: the run-time type checking in Java is separated into the java.lang.reflect package. Interestingly it also adds a notification-based system for concurrency to the root class’s feature set.

C#’s System.Object is similar to Java’s, though without the concurrency thing. Unlike the Java/Foundation root classes, its copy operation (MemberwiseClone()) actually works, creating a shallow copy of the target object.

Things get a bit different when looking at Ruby’s system. The Object class exposes all sorts of functionality: in addition to introspection, it offers the kind of modifications to classes that ObjC programmers would do with runtime functions. It offers methods for “freezing” objects (marking them read-only), “tainting” them (marking them as containing potentially-dangerous data), “untrusting” them (which stops them working on objects that are trusted) and then all the things you might find on NSObject. But there’s a wrinkle. Object isn’t really a root class: it’s just the conventional root for Ruby classes. It is itself a subclass of BasicObject, and this is about the simplest root class of any of the systems looked at so far. It can do equality comparison, message forwarding (which Objective-C supports via the runtime, and NSObject has API for) and the ability to run blocks of code within the context of the receiving object.

C++ provides the least behaviour to its classes: simple constructors that are referenced but not defined can be generated.

It’s useful to realise that even supposedly simple rules like “single responsibility principle” are situated in the context of the software system. Programmers will expect an object with a “single” responsibility to additionally adopt all the responsibilities of the base class, which in something like Foundation can be numerous.

Posted in Foundation, Java, ruby | Leave a comment

More security processes go wrong

I just signed a piece of card so that I could take a picture of it, clean it up and attach it to a document, pretending that I’d printed the document out, signed it, and scanned it back in. I do that about once a year (it was more frequent when I ran my own business, but then I only signed the piece of card once).

Just a little reminder: it’s not having my signature that should be valued, it’s having seen me perform the act of signing. Signatures can easily be duplicated. If you’ve decided that I’m me, and you’ve seen me put my signature to a document, from that moment on you can know that I signed that document. If you didn’t see it, but got a validated statement from a known notary that they saw it, then fair enough. If you didn’t see it, and a notary didn’t see it, then all you know is that you have a sheet of paper containing some words and my signature. This should tell you nothing about how the two came into proximity.

Posted in Authentication, Vulnerability | Comments Off on More security processes go wrong

Could effortless lecturers make everything seem too easy?

From the British Psychological Society blog: Engaging lecturers can breed overconfidence.

The students who’d seen the smooth lecturer thought they would do much better than did the students who saw the awkward lecturer, consistent with the idea that a fluent speaker breeds confidence. In fact, both groups of students fared equally well in the test. In the case of the students in the fluent lecturer condition, this wasn’t as good as they’d predicted. Their greater confidence was misplaced.

I’m speaking at a couple of conferences later this year (iOSDev UK in Aberystwyth in September and NSScotland in Edinburgh in October), and will endeavour to be exactly as exciting as the material deserves: a capability to which my track record can attest.

Posted in psychology, Talk | Comments Off on Could effortless lecturers make everything seem too easy?

Objective-C, dependencies, linking

In the most recent episode of Edge Cases, Wolf and Andrew discuss dependency management, specifically as it pertains to Objective-C applications that import libraries using the Cocoapods tool.

In one app I worked on a few years ago, two different libraries each tried to include (as part of the libraries themselves, not as dependencies) the Reachability classes from Apple’s sample code. The result was duplicate symbol definitions, because my executable was trying to link both (identical) definitions of the classes. Removing one of the source files from the build fixed it, but how could we avoid getting into that situation in the first place?

One way explored in the podcast is to namespace the classes. So Mister Framework could rename their Reachability to MRFReachability, Framework-O-Tron could rename theirs to FOTReachability. Now we have exactly the same code included twice, under two different names. They don’t conflict, but they are identical so our binary is bigger than it needs to be.

It’d be great if they both encoded their dependency on a common class but didn’t try to include it themselves so we could just fetch that class and use it in both places. Cocoapods’s dependency resolution allows for that, and will work well when both frameworks want exactly the same Reachability class. However, we hit a problem again when they want different libraries, with the same names in.

Imagine that the two frameworks were written using different versions of libosethegame. The developers changed the interface when they went from v1.0 to v1.1, and Framework-O-Tron is still based on the older version of the interface. So just selecting the newer version won’t work. Of course, neither does just selecting the older version. Can we have both versions of libosethegame, used by the two different frameworks, without ending up back with the symbol collision error?

At least in principle, yes we can. The dynamic loader, dyld (also described in the podcast) supports two-level namespace for dynamic libraries. Rather than linking against the osethegame library with -losethegame, you could deploy both libosethegame.1.0.0.dylib and libosethegame.1.1.0.dylib. One framework links with -losethegame.1.0, the other links with -losethegame.1.1. Both are deployed, and the fact that they were linked with different names means that the two-level namespace resolves the correct symbol from the correct version of the library, and all is well.

Of course, if you’ve got dynamic libraries, and the library vendor is willing to do a little work, they can just ship one version that supports all previous behaviour, looking at which version of the library the client was linked against to decide what behaviour to provide. Despite Mac OS X providing a versioned framework bundle layout, Apple has never (to my knowledge) shipped different versions of the system frameworks. Instead, the developers use the Mach-O load headers for an executable to find the linked version of their library, and supply behaviour equivalent to that version.

The above two paragraphs do rather depend on being able to use the dynamic linker. We can’t, on iOS, at the moment.

Posted in code-level, tool-support | Leave a comment

When security procedures go bad

My password with my bank may as well be “I can’t remember, can we go through the security questions please?” That’s my answer so many times when they ask, and every time it gets me in via a slightly tedious additional verification step. Losing customers probably represents a greater financial risk to them than fraud on any individual account, so they don’t seem to take the password thing too seriously.

Posted in Uncategorized | Comments Off on When security procedures go bad

Specifications for interchanging objects

One of the interesting aspects of Smalltalk and similar languages including Objective-C and Ruby is that while the object model exposes a hierarchy of classes, consumers of objects in these environments are free to ignore the position of the object in that hierarchy. The hierarchy can be thought of as a convenience: on the one hand, for people building objects (“this object does all the same stuff as instances of its parent class, and then some”). It’s also a convenience for people consuming objects (“you can treat this object like it’s one of these types further up the hierarchy”).

So you might think that -isKindOfClass: represents a test for “I can use this object like I would use one of these objects”. There are two problems with this, which are both expressed across two dimensions. As with any boolean test, the problems are false positives and false negatives.

A false positive is when an object passes the test, but actually can’t be treated as an instance of the parent type. In a lot of recent object-oriented code this is a rare problem. The idea of the Liskov Substitution Principle, if not its precise intent as originally stated, has become entrenched in the Object-Oriented groupthink.

I’ve worked with code from the 1980s though where these false positives exist: an obvious example is “closing off” particular selectors. A parent class defines some interface, then subclasses inherit from that class, overriding selectors to call [self doesNotRecognize:] on features of the parent that aren’t relevant in the subclass. This is still possible today, though done infrequently.

False negatives occur when an object fails the -isKindOfClass: test but actually could be used in the way your software intends. In Objective-C (though neither in Smalltalk[*] nor Ruby), nil _does_ satisfy client code’s needs in a lot of cases but never passes the hierarchy test. Similarly, you could easily arrange for an object to respond to all the same selectors as another object, and to have the same dynamic behaviour, but to be in an unrelated position in the hierarchy. You _can_ use an OFArray like you can use an NSArray, but it isn’t a kind of NSArray.

[*] There is an implementation of an Objective-C style Null object for Squeak.

Obviously if the test is broken, we should change the test. False negatives can be addressed by testing for protocols (again, in the languages I’ve listed, this only applies to Objective-C and MacRuby). Protocols are unfortunately named in this instance: they basically say “this object responds to any selector in this list”. We could then say that rather than testing for an object being a kind of UIView, we need an object that conforms to the UIDrawing protocol. This protocol doesn’t exist, but we could say that.

Problems exist here. An object that responds to all of the selectors doesn’t necessarily conform to the protocol, so we still have false negatives. The developer of the class might have forgotten to declare the protocol (though not in MacRuby, where protocol tests are evaluated dynamically), or the object could forward unknown selectors to another object which does conform to the protocol.

There’s still a false positive issue too: ironically protocol conformance only tells us what selectors exist, not the protocol in which they should be used. Learning an interface from a protocol is like learning a language from a dictionary, in that you’ve been told what words exist but not what order they should be used in or which ones it’s polite to use in what circumstances.

Consider the table view data source. Its job is to tell the table view how many sections there are, how many rows there are in each section, and what cell to display for each row. An object that conforms to the data source protocol does not necessarily do that. An object that tells the table there are three sections but crashes if you ask how many rows are in any section beyond the first conforms to the protocol, but doesn’t have the correct dynamic behaviour.

We have tools for verifying the dynamic behaviour of objects. In his 1996 book Superdistribution: Objects as Property on the Electronic Frontier, Brad Cox describes a black box test of an object’s dynamic behaviour, in which test code messages the object then asserts that the object responds in expected ways. This form of test was first implemented in a standard fashion, to my knowledge, in 1998 by Kent Beck as a unit test.

Unit tests are now also a standard part of the developer groupthink, including tests as specification under the name Test-Driven Development But we still use them in a craft way, as a bespoke specification for our one-of-a-kind classes. What we should really do is to make more use of these tests: substituting our static, error-prone type tests for dynamic specification tests.

A table view does not need something that responds to the data source selectors, it needs something that behaves like a data source. So let’s create some tests that any data source should satisfy, and bundle them up as a specification that can be tested at runtime. Notice that these aren’t quite unit tests in that we’re not testing our data source, we’re testing any data source. We could define some new API to test for satisfactory behaviour:

- (void)setDataSource: (id <UITableViewDataSource>)dataSource {
  NSAssert([Demonstrate that: dataSource satisfies: [Specification for: @protocol(UITableViewDataSource)]]);
  _dataSource = dataSource;
  [self reloadData];
}

But perhaps with new language and framework support, it could look like this:

- (void)setDataSource: (id @<UITableViewDataSource>)dataSource {
  NSAssert([dataSource satisfiesSpecification: @specification(UITableViewDataSource)]);
  _dataSource = dataSource;
  [self reloadData];
}

You could imagine that in languages that support design-by-contract, such as Eiffel, the specification of a collaborator could be part of the contract of a class.

In each case, the expression inside the assertion handler would find and run the test specification appropriate for the collaborating object. Yes this is slower than doing the error-prone type hierarchy or conformance tests. No, that’s not a problem: we want to make it right before making it fast.

Treating test fixtures as specifications for collaboration between objects, rather than (or in addition to) one-off tests for one-off classes, opens up new routes for collaboration between the developers of the objects. Framework vendors can supply specifications as enhanced documentation. Framework consumers can supply specifications of how they’re using the frameworks as bug reports or support questions: vendors can add those specifications to a regression testing arsenal. Application authors can create specifications to send to contractors or vendors as acceptance tests. Vendors could demonstrate that their code is “a drop-in replacement” for some other code by demonstrating that both pass the same specification.

But finally it frees object-oriented software from the tyranny of the hierarchy. The promise of duck typing has always been tempered by the dangers, because we haven’t been able to show that our duck typed objects actually can quack like ducks until it’s too late.

Posted in documentation, OOP, software-engineering, TDD, tool-support | 1 Comment

Can Objective-C be given safe categories?

That was the subject of this lunchtime’s vague thinking out loud. The problems with categories are well-known: you can override the methods already declared on a class, or the methods provided in another category (and therefore another category can replace your implementations too). Your best protection is to use ugly wartifying prefixes in the hope that your bewarted method names don’t collide with everybody else’s bewarted method names.

A particular problem with categories, and one that’s been observed in the wild, is when you add a method in a category that is, at some later time, added to the original implementation of the class itself. Other consumers of the class (including the framework it’s part of) may be expecting to work with the first-party implementation, not your substitution. If the first-party method has a different binary interface to yours (e.g. one of you returns a primitive value and the other a struct), as happened to a lot of people with NSArray around the end of the 1990s, prepare to start crashinating.

Later implementations of similar features in other languages have avoided this problem by refusing to add methods that already exist, and by ensuring that even if multiple extensions define the same method they can all coexist and the client code expresses exactly which one it’s referring to. Can we add any of this safety to Objective-C?

Partially. We could design a function for adding a collection of methods from a “category” to a class at runtime, that only adds them if the class doesn’t already implement them. class_addCategory() shows what this might look like, but it only supports non-struct-returning instance methods.

If class_addCategory(target, source, NO) succeeds, then the methods you were trying to add did not exist on the target class before you called the function. However, you cannot be sure that they weren’t being added while your call was in progress, and you can’t know later that they weren’t clobbered by someone else at some point between successfully adding the methods and using them. Also, if class_addCategory() fails, you may find the only reasonable course of action is to not use the methods you were trying to add: the only thing you know about their implementation is that it either doesn’t exist or isn’t the one you were expecting. This is at odds with a hypothetical purist notion of Object-Oriented Programming where you send messages to objects and don’t care what happens as a result.

There are plenty of ways to work around the limitations of categories: composition is the most likely to succeed (more likely than subclassing, which suffers the same collision problem as a later version of the superclass might try to define a method with the same name as one you’ve chosen, which you’re now clobbering). It doesn’t let you replace methods on a class—a tool that like most in the programmer’s utility belt is both occasionally useful and occasionally abuseful.

Coda

I should point out that I’m not a fan of taking away the potentially dangerous tools. Many people who see the possibility for a language feature to be abused argue that it should never be used or that languages that don’t offer it should be preferred. This is continuum-fallacy nonsense, to which I do not subscribe. Use whatever language features help you to produce a working, comprehensible, valuable software system: put in whatever protections you want to guard against existing or likely problems.

Posted in code-level, OOP | Comments Off on Can Objective-C be given safe categories?

APPropriate Behaviour is almost done

I just pushed another update to APPropriate Behaviour, my work on the things programmers do that aren’t programming. There’s some refinement to the existing material to be done, and a couple of short extra chapters to finish and add. But then it will be complete!

The recommended price of APPropriate Behaviour is $20. While it’s been under development, I’ve allowed readers interested in a sneak peak to buy APPropriate Behaviour at any price above $5. Once the final chapters are in place, the recommended price will remain $20 but the minimum price will be increasing. If you’ve been pondering buying it but haven’t yet, I recommend you do so now to get a bargain. Even if you buy it while I’m still working on it, you’ll get free updates for life as I add new material and make corrections.

As a little taster of things to come, the two remaining chapters are:

  • The ethics of making software
  • The philosophy of making software

Can’t wait to see what that means? Neither can I!

Posted in advancement of the self, books | Leave a comment

As the Kaiser Chiefs might say: Ruby ruby ruby n00bie

Imagine someone took the training wheels off of Objective-C. That’s how I currently feel.

Bike with Training Wheels: image credit Break

I’ve actually had a long—erm, not quite “love-hate”, more “‘sup?-meh”—relationship with Ruby. I’ve long wanted to tinker but never really had a project where I could make it fit; I did learn a little about Rails a couple of years back but didn’t then get to put it into practice. Recently I’ve been able to do some Real Work™ with Ruby, and wanted to share the experience.

Bear in mind that when I say I’ve been working with Ruby, I mean that I’ve been writing Objective-C in Ruby. This becomes clear when we see one of the problems I’ve been facing: I couldn’t work out how to indicate that a variable exposes some interface, until I realised I didn’t need to. Ruby takes the idea of duck typing much further than Objective-C does: using Ruby is much more like Smalltalk in that you don’t care what an object is, you care what it does. Currently no tools really support that way of working (and so Stockholm Syndrome-wielding developers will tell you that you don’t need such tools; just vi and a set of tests); the first warning I get when I’ve made a mistake is usually an exception backtrace. Something I had to learn quite quickly is that Ruby and Objective-C have different ideas of nil: Ruby behaves as the gods intend and lets you put nil into collections; but Objective-C behaves as the gods intend and lets you treat nil as a null object.

The problems I’ve been facing have largely involved learning how things are conventionally done. One example is that a library I was using took a particular parameter and treated it as a constant. Apparently Matz is a big fan of Fortran, but only early hipster Fortran before they sold out and added implicit none (around the time they fired their bass player and started playing the bigger venues). So Ruby provides its own implicit convention: constants have to be named starting with an uppercase. Otherwise you get told this:

wrong constant name parameter-value

Erm, that’s it. Not “you should try calling it ParameterValue“, or “constants must start with a capital letter”. Not even “this is not a good name for a constant”; who else interpreted that as “you gave the name of the wrong constant”? I think I’ve been spoiled by the improvements to the clang diagnostics over the last couple of years, but I found some of Ruby’s messages confusing and unhelpful. This is often the case with software that relies on convention: once you know the conventions you can go really fast, but when you don’t know them you feel like you’re being ignored or that it’s being obtuse.[*]

[*] When I asked for help on this issue I was told I suggest you pick of[sic] a good Ruby book or watch some Ruby tutorials on YouTube; you’ll be pleased to know that the interpreter wasn’t the only ignorant or obtuse tool I had to deal with.

These are very neophyte problems though, and once I got past them I found that I was able to make good progress with the language. I was using LightTable and RubyMine for editing, and found that I could work really quickly with a combination of those editors and irb. Having an interactive environment or a REPL is amazing for trying out little things, particularly when you’re new at a language and don’t know what’s going to work. It’s a bit cumbersome for more involved tests, but the general execute-test cycle is much faster than with Objective-C.

Speaking of tests, I know that if you ask four Ruby developers how to write unit tests you’ll get six different answers and at least eighteen of them will have moved on to Node.JS. I’ve been using Mini::Test, as it’s part of the standard library so involved the least configuration to get going.

I also took the opportunity to install MacRuby and have a go at building a Mac app, using Cocoa Bindings on the UI side to work with controllers and models that I’d written in Ruby. This isn’t the first exposure I’ve had to a bridged environment: I’ve done a lot of Perl-Cocoa with CamelBones, the PerlObjCBridge and ObjectiveFramework. MacRuby isn’t like those bridges though, in that (as I understand it) MacRuby builds Ruby’s object model on top of NSObject and the Objective-C runtime so Ruby objects actually are ObjC objects. It means there’s less manual gluing: e.g. in Perl you might do:

my $string = NSString->alloc->initWithCString_encoding_("Hello", NSUTF8StringEncoding);

In MacRuby that becomes:

string = "Hello"

That’s not to say there’s no boilerplate. I found that by-return references need the creation of a Pointer object on the Ruby side to house the pointer to the object reference, which looks like this:

error = Pointer.new(:object)
saveResult = string.writeToFile path, atomically: false, encoding: NSUTF8StringEncoding, error: error

For a long time, I’ve thought that there would be mileage in suggesting programmers use a different language than Objective-C for building applications in Cocoa, relying on ObjC as the systems language. Ruby could be that thing. The object models are very similar, so there isn’t a great deal of mind-twisting going on in exposing Objective-C classes and objects in Ruby. There’s a lot less “stuff you do that shuts the compiler up”, though ObjC has seen a reduction in that itself of late it still relies on C and all of its idiosyncrasies. Whether it’s actually better for some developers, and if so for whom, would need study.

Summarising, Ruby feels a lot like Objective-C without the stabilisers. You can work with objects and methods in a very similar way. The fast turnaround afforded by having an interactive shell and no compile-link waiting means you can go very quickly. The fact that you don’t get the same up-front analysis and reporting of problems means you can easily drive into a wall at full tilt. But at least you did so while you were having fun.

Posted in OOP, ruby | Comments Off on As the Kaiser Chiefs might say: Ruby ruby ruby n00bie