Skip to content

On home truths in iOS TDD

The first readers of Test-Driven iOS Development (currently available in Rough Cuts form on Safari Books Online: if you want to buy a paper/kindle/iBooks editions, you’ll have to wait until it enters full production in a month or so) are giving positive feedback on the book’s content, which is gratifying. Bar last minute corrections and galley proof checking, my involvement with the project is nearly over so it’s time for me to reflect on the work that has dominated my schedule for over a year.

As explained in the book’s front matter, I chose to give all of the examples in the book and accompanying source code using OCUnit. As the BBC might say, “other unit test frameworks are available”. Some of the alternative frameworks are discussed in the book, so interested readers can try them out for themselves.

What made OCUnit the correct choice—put it a different way, what made OCUnit the choice I made? It’s the framework that’s shipped with Xcode, so anyone who might want to try out unit testing can pick up the book and give it a go. There are no third-party dependencies to become unsupported or change beyond all recognition—though that does occasionally happen to Xcode. File-New Project…, include unit tests, and you’re away, following the examples and trying out your own things.

Additionally, the shared body of knowledge in the Cocoa development community is greatest when it comes to OCUnit. Aside from people who consider automated testing to be teh suck, plenty of developers on Mac, iOS and other platforms have got experience using OCUnit or something very much like it. Some of those people have switched to other frameworks, but plenty are using OCUnit. There’s plenty of experience out there, and plenty of help available.

The flip side to this is that OCUnit doesn’t represent the state of the art in testing. Far from it: the kit was first introduced in 1998, and hasn’t changed a great deal since. Indeed many of the alternatives we see in frameworks like GHUnit and Google Toolkit for Mac are really not such great improvements, adding some extra macros and different reporting tools. Supporting libraries such as OCHamcrest and OCMock give us some additional features, but we can look over the fence into the neighbouring fields of Java, ruby and C# to see greater innovations and more efficient testers.

Before you decide to take the book out of your Amazon basket, let me assure you that learning TDD via OCUnit is not wasted effort. The discipline of red-green-refactor, the way that writing tests guides the design of your classes, the introduction of test doubles to remove dependencies in tests: these are all things that (I hope) the book can teach you, and that you can employ whether you use OCUnit or some other framework. And, as I said, there’s plenty of code out there that is in an OCUnit harness. It’s not bad, it could be better.

So what are the problems with OCUnit?

  • repetition. Every time you write STAssert, you’re saying two things. Firstly, “hey, I’m using OCUnit”, which isn’t really useful information. Second, “what’s coming up is a test, read on to find out what kind of test”. Then you finally get to the end of the macro where you reveal what it is you’re going to do. This is the important information, but we bury it in the middle of the line behind some boilerplate.

    Imagine, instead, a hypothetical language where we could send messages to arbitrary expressions (ok that exists, but imagine it’s objc). Then you could do [[2+2 should] equal: 4]; which more closely reflects our intention.

  • repetition. In the same way that STAssert is boilerplate, so is sub classing SenTestCase and writing -(void)test at the beginning of every test method. It gives you no useful information, and hides the actual data about the test behind the boilerplate.

    Newer test frameworks in languages like C# and Java use the annotation features of those languages to take the fact that a method is a test out of its signature and make it metadata. ObjC doesn’t support annotations, so we can’t do that. But take a look at the way CATCH tests are marked up. You indicate that something is a test, and the fact that this means the framework needs to generate an objective-c++ class and call a method on it is encapsulated in the framework’s implementation.

  • repetition. You might think that there’s a theme developing here :-). If you write descriptive method names, you might have a test named something like -testTheNetworkConnectionIsCleanedUpWhenADownloadFails. Should that test fail, you’re told what is going wrong: the network connection is not cleaned up when a download fails.

    So what should you write in the mandatory message parameter all of the STAssert…() macros require? How about @"the network connection was not cleaned up when a download failed"? Not so useful.

  • organisation. I’ve already discussed how OCUnit makes you put tests into particular classes and name them in particular ways. What if you don’t want to do that? What if you want to define multiple groups of related tests in the same class, in the way BDD practitioners do to indicate they’re all part of the same story? What if you want to group some of the tests in one of those groups? You can’t do that.

I’m sure other people have other complaints about OCUnit, and that yet other people can find no fault with it. In this post I wanted to draw attention to the fact that there’s more than one way to crack a nut, and the vendor-supplied nutcracker is useful though basic.

{ 5 } Comments

  1. Stewart Gleadow | February 7, 2012 at 9:41 pm | Permalink

    Nice write up. OCUnit has really progressed in the last few years and is quite a usable testing framework now.

    Of your 4 problems with OCUnit, I think that 1, 3 and 4 are well solved by using Kiwi… but I have been trying to use OCUnit in my examples for the same reasons that you outlined above.

    A couple of OCUnit complaints off the top of my head:

    – running application tests from the command line still requires a hack to the shell script as far as I know

    – why differentiate logic/application tests? Application tests don’t sound very unit-like. It should be able to decide automatically based on whether UIKit is linked

    – test output integration into the IDE is not nearly up to JUnit/

    – running a single test or set of tests over and over while TDDing is slow, I want a keyboard shortcut to run the test that currently has focus in the IDE

  2. Anthony | February 8, 2012 at 8:40 am | Permalink

    It’s possible to write this when using Kiwi:
    [[theValue(2+2) should] equal: theValue(4)];

  3. Graham | February 8, 2012 at 10:15 am | Permalink

    Thanks to both for describing that Kiwi addresses many of the problems I’ve listed with the readability of OCUnit tests.

    Stewart: I’d blanked the app/logic distinction out of my brain 🙂 and in the book I explain that I don’t care for it and remove all the boilerplate and preprocessor cruft associated with it. I can see the point of ‘application tests’ as an integration test scheme, and OCUnit is perfectly capable of running integration tests. It’s confusing for people unfamiliar with Apple’s terminology though, so perhaps they should have “unit test” and “integration test” targets (as well as “system test” targets that harness UIAutomation instruments…).

    Your points on IDE integration, particularly “run this test” one, are good and would help a lot. To the Radars!

  4. Vincent Tourraine | February 8, 2012 at 3:12 pm | Permalink

    I’m using Cedar (http://pivotal.github.com/cedar/) now, and I really enjoy it.

    It can be coupled with OCUnit, so I simply hit cmd+U to run my tests, and the syntax is really nice, IMO.

    For instance:
    object1.name should equal(object2.name);
    object1.name should equal(@”a”);
    object1.items should contain(1);

    Looking forward to your book.

  5. Phil Nash | February 15, 2012 at 4:44 pm | Permalink

    Thanks for mentioning Catch (in the book too!).

    I feel obliged to add that Catch also addresses most of your other concerns in this post (as well as some of the other commenters). It doesn’t (yet) support matchers, and it doesn’t strictly support BDD-style syntax.
    But matchers are coming.
    Assertions are much more natural already.
    You can divide test cases in sections (and sub-sections – nested to any level).
    Test names are free-form text, with an optional description (soon to be “even more optional”).

    Oh and it’s all in a single header file – no frameworks to add – so the smallest possible increment of overhead over using OCUnit.

    I’m just now working on beefing up the iOS support too.

    For anyone interested see: http://www.levelofindirection.com/journal/2010/12/28/unit-testing-in-c-and-objective-c-just-got-easier.html

    And:
    http://www.levelofindirection.com/journal/2011/5/27/unit-testing-in-c-and-objective-c-just-got-ridiculously-easi-1.html

{ 2 } Trackbacks

  1. Macintosh tdd | Bgmiami | March 31, 2012 at 8:22 am | Permalink

    […] On home truths in iOS TDD — Secure Mac ProgrammingFeb 7, 2012 … Aside from people who consider automated testing to be teh suck, plenty of developers on Mac, iOS and other platforms have got experience … […]

  2. […] http://blog.securemacprogramming.com/2012/02/on-home-truths-in-ios-tdd/ […]