Thoughts on Tech Conferences

This post is being, um, posted from the venue for GOTO Copenhagen 2012. It’s the end result of a few months of reflection on what I get out of conferences, what I want to get out of conferences, what I put into (and want to put into) conferences and the position of tech conferences in our industry. I’ve also been discussing things a lot with my friends and peers; I’ve tried to attribute specific quotes where I remember who said them but let it be known that many people have contributed to the paragraphs below in many different ways. I’ll make it clear at the outset that I’m talking about my experience at independent commercial and non-profit tech conferences, not scientific conferences (of which I have little experience) or first-party events like WWDC (which are straightforward marketing exercises).

Conference speakers

My favourite quote on this subject is courtesy of Mike; I remember him saying it in his MDevcon keynote this year but I’m also fairly sure he’s said the same thing earlier:

The talks at a conference are only there so that you can claim the ticket cost as an expense.

We’re in a knowledge economy; but knowledge itself is not of any value unless it’s applied. That means it’s not the people who tell other people what’s going on who’re are doing the most important work; that’s being done by the people who take this raw knowledge, synthesise it into a weltanschauung – a model of how the world works – and then make things according to that model. Using an analogy with the economy of physical things, when we think of the sculpture of David in Florence we think of Michelangelo, the sculptor, not of the quarry workers who extracted the marble from the ground. Yes their work was important and the sculpture wouldn’t exist without the rock, but the most important and valuable contribution comes from the sculptor. So it is in the software world. Speakers are the quarry workers; the marble hewers, providing chunks of rough knowledge-stuff to the real artisans – the delegates – who select, combine and discard such knowledge-stuff to create the valuable sculptures: the applications.

Michelangelo's David, source: Wikimedia Commons.

Conference speakers who believe that the value structure is the other way around are deluding themselves. Your talk is put on at the conference to let people count the conference as a work expense, and to inspire further discussion and research among the delegates on the topic you’re talking about. It’s not there so that you can promote your consultancy/book/product, or produce tweet worthy quotes, or show off how clever you are. Those things run the gamut from “fringe benefits” to “deleterious side effects”.

As an aside, the first time I presented at a Voices That Matter conference I was worried due to the name; it sounds like the thing that matters at this conference is the speakers’ voices. In fact I suspect there is some of that as many of the presenters have books published by the conference hosts, but it’s a pretty good conference covering a diverse range of topics, with plenty of opportunities to talk to fellow delegates. And IIRC all attendees got an “I am one of the voices that matter” sticker. Anyway, back to the topic at hand: thus do we discover a problem. Producing a quality conference talk is itself knowledge work, that requires careful preparation, distillation and combination of even more raw knowledge-stuff. It takes me (an experienced speaker who usually gets good, but not rave, reviews) about three days to produce a new one hour talk, a roughly 25:1 ratio of preparation:delivery. That’s about a day of deciding what to say and what to leave out, a day of designing and producing materials like slides, handouts and sample code, and a day of practising and editing. Of course, that’s on top of whatever research it was that led me to believe I could give the talk in the first place. The problem I alluded to at the start of the last paragraph is this: there’s a conflict between acknowledging that the talks are the bricks-and-mortar of the conference rather than the end product, and wanting some return on the time invested. How that conflict’s resolved depends on the personal values of the individual; I won’t try to speak for any of my peers here because I don’t know their minds.

The conference echo chamber

That’s not my phrase; I’ve heard it a lot and can track my most recent recollection to @secwhat’s post Conference Angst. Each industry’s conferences has a kind of accepted worldview that is repeated and reinforced in the conference sessions, and that only permits limited scrutiny or questioning – except for one specific variety which I’m coming onto later. As examples, the groupthink in indie Mac/iOS conferences is “developers only need developer features that have been blessed by Apple”. There’s recently been significant backlash to the RubyMotion framework, as there usually is when a new third-party abstraction for iOS appears. But isn’t abstraction a good thing in software engineering? The truth is, of course, that there are more things in heaven and earth than are dream’t in Apple’s philosophy.

The information security groupthink is that information security is working. Shocking though it may sound, that’s far from obvious, evident or even demonstrable. Show me the blind test where similar projects were run with different levels of info sec engagement and where the outcome was significantly different. Demonstrate how any company’s risk profile has changed since last year. Also, show me an example of security practitioners being ahead of the curve, predicting and preparing for a new development in the field: where were the talks on hacktivism before Anonymous or Wikileaks?

One reason that the same views are repeated over multiple conferences is that the same circuit of speakers travels to all of the conferences. I’m guilty of perpetuating that myself, being (albeit unintentionally in one year) one of the speakers in the iOS circuit. And when I’ve travelled to Seattle or Atlanta or Copenhagen or Aberystwyth, I’ve always recognised at least a few names in the speaker line-up. [While I mentioned Aberystwyth here, both iOSDevUK and NSConf take steps to address the circuit problem. iOSDevUK had a number of first-time speakers and a “bar camp” where people could contribute their own talks. NSConf has the blitz talks which are an accessible way to get a large number of off-circuit speakers, and on one occasion ran a whole day of attendee-contributed sessions called NSConf Mini. When you give people who don’t normally present the opportunity to do so, someone will step up.]

I mentioned before that the echo chamber only permits limited scrutiny, and that comes in the form of the “knowing troll” talk. Indeed at GOTO there’s a track on the final day called “Iconoclasm”, which is populated solely with this form of talk. Where the echo chamber currently resounds to the sound of , it’s permitted to deliver an “ sucks” talk. This will usually present a straw man version of and list its failings or shortcomings. That’s allowed because it actually reinforces – real-world examples are rarely anything like as bad as the straw man version, therefore isn’t really that bad. This form of talk is often a last-session-of-the-day entry and doesn’t really lead people to challenge their beliefs. What happens later is that when everyone moves on to the next big thing, the “ sucks” talks will become the main body of the conference and “<X+1> sucks” will be the new troll talk.

Conferences

Weirdly, while the word conference means a bringing together of people to talk, coming from the same root as “conversation”, many conferences are designed around a one-way flow of words from the speakers to the delegates. Here’s the thing with that. As I said in my keynote talk at MDevcon, we learn from each other by telling and listening to stories. Terry Pratchett, Jack Cohen and Ian Stewart even went as far as to reclassify humans as pan narrans, the storytelling chimpanzee. Now if you’ve got M speakers and 10M<N<100M delegates, then putting a sequence of speakers up and listening to their stories gets you a total of M stories. Letting the N delegates each share their stories, and then letting each of the N-1 other delegates share the stories that the first N stories reminded them of, and so on, would probably lead to a total of N! stories if you had the time to host that. But where that does happen, it’s usually an adjunct to the “big top” show which is the speaker series. [And remember: if you’ve got C conferences, you don’t have C*M speakers, you have M+ε speakers.]

There’s one particular form of wider participation that never works well, and that’s to follow a speaker session with Q&A. Listen carefully to the questions asked at the next Q&A you’re in, and you’ll find that many are not questions, but rhetorical statements crafted to make the “asker” appear knowledgable. Some of those questions that are questions are rhetorical land mines with the intent of putting the speaker on the back foot, again to make the asker seem intellectually talented. Few of these questions will actually be of collective value to the plenus, so there’s not much point in holding the Q&A in front of everyone.

Speaker talks are only one way to run a session, though. Panels, workshops and debates all invite more collaboration than speaker sessions. They’re also much more difficult to moderate and organise, so are rarely seen: many conferences have optional days that are called “workshops” but in reality are short training courses run by an invited speaker. In the iOS development world, lab sessions are escaping the confines of WWDC and being seen at more independent conferences. These are like one-on-one or few-on-few problem solving workshops, which are well focussed and highly collaborative but don’t involve many people (except at Voices that Matter, where they ran the usability workshops on the stage in front of the audience). A related idea being run at GOTO right now, which I need to explore, is a whole track of pair programming sessions. The session host chooses a technology and a problem, and invites delegates onto the stage to work through the challenge with the host in a pair-programming format. That’s a really interesting way to attract wider participation; I’ll wait until I’ve seen it in action before reaching an opinion on whether it works.

There’s another issue, that requires a bit more setup to explain. Here’s a Venn diagram for any industry with a conference scene; the areas are indicative rather than quantitative but they show the relation between:

  • the population of all practitioners;
  • the subsection of that population that attends conferences; and
  • the subsection of that population that speaks at conferences.

Conference Venn diagram

So basically conferences scale really badly. Even once we’ve got past the fact that conferences are geared up to engage the participation of only a handful of their attendees, the next limiting factor is that most people in [whatever industry you’re in] aren’t attending conferences. For the stories told at a conference (in whatever fashion) to have the biggest impact on their industry, they have to break the confines of the conference. This would traditionally, in many conferences, involve either publishing the proceedings (I’ve not heard of this happening in indie tech conferences since the NATO conferences of 1968-9, although Keith Duncan is one of a couple of people to mention to me the more general idea of a peer-reviewed industry journal) or the session videos (which is much more common).

To generate the biggest impact, the stories involved must be inspiring and challenging so that the people who watched them, even those who didn’t attend the conference, feel motivated to reflect on and change the way they work, and to share their experiences (perhaps at the same conference, maybe elsewhere). Before moving on to a summary of everything I’ve said so far, I’ll make one more point about the groups drawn on the Venn diagram. Speakers tend to be specialists (or, as Marcus put it in his NSConf talk, subject matter experts) in one or two fields; that’s not surprising given the amount of research effort that goes into a talk (described above). Additionally, some speakers are asked to conferences because they have published a book on the topic the convenor wishes them to speak on; that’s an even longer project of focussed research. This in itself is a problem, because a lot of the people having difficulty with their work are likely to be neophytes, but apparently we’re not listening to them. We listen to self-selected experts opining on why everyone needs to take security/TDD/whatever seriously and why that involves retaining the experts’ consultancy service: we never listen to the people who can tell us that after a month of trying this Objective-C stuff still doesn’t make sense. These are the people who can give us insight into how to improve our practice, because these are the people reminding the experts (and indeed the journeymen) of the problems they had when they’d been at this for a month. They tell us about the issues everyone has, and give us ideas on how we can fix it for all (future) participants.

Conference goers, then, get the benefit of a small handful of specialists: in other words they have a range of experience to call on (vicariously) that is both broad and deep. Speakers of course have the same opportunity, though don’t always get to take full advantage of the rest of a conference due to preparation, equipment tests, post-talk question sessions and the like. The “non-goers” entry in the diagram represents a vast range of skills and experiences, so it’s hard to find any one thing to say about them. Some will be “distance delegates”, attending every conference by purchasing the videos, transcripts or other materials. Some will absorb information by other means, including meet-ups, books, blogs etc. And some will be lone coders who never interact with anyone in their field. Imagine for a moment that your goal in life is to apply the Boy Scout Rule (which I’m going to attribute again to @ddribin because I can’t remember who he got it from; Uncle Bob probably) to your whole industry. Your impact on $thing_you_do will be to leave the whole field, the whole practice a bit better than it was when you got here. (If that really is your goal, then skip the imagination part for a bit.)

It seems to me that the best people to learn from are the conference delegates (who have seen a wide section of the industry in considerable depth) and the best people to transfer that knowledge to are, well, everybody.

Summary of the current position

Conferences are good. I don’t want people to think I’m hating on conferences. They’re enjoyable events, there are plenty of good ones, there’s an opportunity to learn things, and to see fresh perspectives on many aspects of our industry. They’re also more popular than ever, with new events appearing (and selling out rapidly) every year. However, these perspectives often have an introspective, echo chamber quality. We’re often listening to a small subset of the conference delegates, and if you integrate over multiple conferences you find the subset gets relatively smaller because it’s the same people presenting all the time. Most delegates will not get the benefit of listening to all of the other delegates, which means they’re missing out on engaging with some of the broadest experience in the industry. Most of the practitioners in your corner of the industry probably don’t attend any conferences anyway; there aren’t enough seats for that to work.

The ideal tech conference

OK, I am very clearly lying here: this isn’t the ideal tech conference, it’s my ideal tech conference. In my world, those are the same thing. PerfectConf features a much more diverse portfolio of speakers. In the main this is achieved exactly the way that Appsterdam does it; by offering the chance to speak to anyone who’ll take it, by looking for things that are interesting to hear about rather than accomplished or expert speakers to say it, and by giving novice speakers the chance to train with the experts before they go in front of the stage. Partly this diversity is achieved by allowing people who aren’t comfortable with speaking the opportunity to host a different kind of session, for example a debate or a workshop.

In addition to engaging session hosts who would otherwise be apprehensive about presenting, we get to hear about the successes and tribulations encountered by the whole cohort of delegates. At least one session would be a plenary debate, focussed on a problem that the industry is currently facing. This session has the modest aim of discovering a solution to the problem to move the industry as a whole forward. Another way in which diversity is introduced into the conference is by listening to people outside of our own sector. If infosec is having trouble getting budget for its activities, perhaps they ought to invite more CFOs or comptrollers to its conferences to discuss that. If iPhone app developers find it hard to incorporate concurrency into their application designs, they could do worse than to listen to an Erlang or Occam expert. Above all, the echo chamber would be avoided; session hosts would be asked to challenge the perceived industry status quo.

I’ve long thought that if a talk of mine doesn’t annoy at least one member of the audience then I haven’t said anything useful; a former manager of mine said “if we both think the same way about everything then one of us is redundant”. This way of thinking would be codified into the conference. Essentially, what I’m talking about is the death of the thought leader (or “rock star”). Rather than having one subject matter expert opining on how everyone should think about security, UX, marketing, or whatever, PerfectConf encourages the community to work together like a slime mould, allowing the collective motion of all of the members to explore all opportunities and options and select the best one by communicating freely across the colony.

Slime mold solving a maze (photo: Nature)

Finally, PerfectConf proceedings are published as soon as practical; not just the speaker sessions but the debates too. Where the plenus reaches a consensus, the consensus decision becomes available for all those people who couldn’t make it to the conference of to discover, consider, and potentially adopt or react to. Unfortunately I’m not a conference organiser.

BrowseOverflow as a Code Kata

This article was originally posted over at InformIT.

My goal in writing Test-Driven iOS Development was to take readers from not knowing how to write a test for their iOS apps, to understanding the TDD workflow and how it could work for them. That mirrored the journey that I had taken in learning about test-driven development, and that had led me to wanting to write a book to share what I’d learned with my peers.

This has an interesting effect on the structure of the book. Not all of the sample code from the BrowseOverflow is shown (though it’s all available on GitHub). This isn’t an accident we made in the editorial process. It’s a feature: for any test shown in the book, there are numerous different ways you could write application code that would all be just as good. Anything that causes the test to pass, and doesn’t make the other tests fail, is fine.

Just as the many-worlds model in quantum theory says that there are many branches of the universe that are created every time a decision is made, so there’s a “many-BrowseOverflows” model of Test-Driven iOS Development. Every time a test is written, there are many different solutions to passing the test, leading to there being multiple potential BrowseOverflows. The code that you see on GitHub is just one possible BrowseOverflow, but any other code that satisfies the same tests is one of the other possible BrowseOverflows.

This means that you can treat the book like a kata: the Japanese martial art technique of improving a practice by repeating it over and over. The first time you read Test-Driven iOS Development, you can choose to follow the example code very closely. Where the code isn’t given in the book, you might choose to look at the source code to understand how I solved the problems posed by the tests. But the end of the book is not the end of the journey: you can go back, taking the tests but implementing all of the app code yourself. You can do this as many times as you want, trying to find new ways to write code that produces a BrowseOverflow app.

Finally, when you’re more confident with the red-green-refactor way of working, you can write a BrowseOverflow app that’s entirely your own creation. You can define what the app should do, create tests that express those requirements and write the code to implement it however you like. This is a great way to test out new ways of working, for example different test frameworks like GHUnit or CATCH. It also lets you try out different ways of writing the application code: you could write the same app but trying to use more blocks, or try to use the smallest number of properties in any class, or any other challenge you want to set yourself. Because you know what the software should be capable of, you’re free to focus on whatever skill you’re trying to exercise.

Test-Driven iOS Development

Here it is, after more than a year in the making, the book that they really did want you to read! Test-driven IOS Development (Developer’s Library) (affiliate link) has finally hit the stores[*].

I wrote this book for the simple reason that it didn’t exist. Like Professional Cocoa Application Security (Wrox Professional Guides) (another affiliate link), I knew that the topic was something many people were interested in, including myself. I wanted a single place I could go to find out about using Xcode for writing unit tests, and how the Test-Driven Development approach could be applied to writing Cocoa code with Objective-C.

Well, it turned out that this place didn’t exist, so I tried my best to create it. In the book you’ll see how the BrowseOverflow app was built entirely in a test-first fashion, using OCUnit and Xcode 4. Along the way, I hope you’ll see how TDD can be useful for your own projects.

If you’re interested in finding out more about test-driven development for iOS apps, I’ll be giving a talk on the subject at iOSDev UK in Aberystwyth in July. And of course I’m happy to field questions about the book or about TDD here or over on Twitter. And of course my unit testing video training course is still available from iDeveloper.tv.

Happy testing!

[*]The availability appears to vary by country and format, so if it isn’t in your favourite (e)bookstore yet just be patient and keep refreshing that page like it’s the WWDC site.

On home truths in iOS TDD

The first readers of Test-Driven iOS Development (currently available in Rough Cuts form on Safari Books Online: if you want to buy a paper/kindle/iBooks editions, you’ll have to wait until it enters full production in a month or so) are giving positive feedback on the book’s content, which is gratifying. Bar last minute corrections and galley proof checking, my involvement with the project is nearly over so it’s time for me to reflect on the work that has dominated my schedule for over a year.

As explained in the book’s front matter, I chose to give all of the examples in the book and accompanying source code using OCUnit. As the BBC might say, “other unit test frameworks are available”. Some of the alternative frameworks are discussed in the book, so interested readers can try them out for themselves.

What made OCUnit the correct choice—put it a different way, what made OCUnit the choice I made? It’s the framework that’s shipped with Xcode, so anyone who might want to try out unit testing can pick up the book and give it a go. There are no third-party dependencies to become unsupported or change beyond all recognition—though that does occasionally happen to Xcode. File-New Project…, include unit tests, and you’re away, following the examples and trying out your own things.

Additionally, the shared body of knowledge in the Cocoa development community is greatest when it comes to OCUnit. Aside from people who consider automated testing to be teh suck, plenty of developers on Mac, iOS and other platforms have got experience using OCUnit or something very much like it. Some of those people have switched to other frameworks, but plenty are using OCUnit. There’s plenty of experience out there, and plenty of help available.

The flip side to this is that OCUnit doesn’t represent the state of the art in testing. Far from it: the kit was first introduced in 1998, and hasn’t changed a great deal since. Indeed many of the alternatives we see in frameworks like GHUnit and Google Toolkit for Mac are really not such great improvements, adding some extra macros and different reporting tools. Supporting libraries such as OCHamcrest and OCMock give us some additional features, but we can look over the fence into the neighbouring fields of Java, ruby and C# to see greater innovations and more efficient testers.

Before you decide to take the book out of your Amazon basket, let me assure you that learning TDD via OCUnit is not wasted effort. The discipline of red-green-refactor, the way that writing tests guides the design of your classes, the introduction of test doubles to remove dependencies in tests: these are all things that (I hope) the book can teach you, and that you can employ whether you use OCUnit or some other framework. And, as I said, there’s plenty of code out there that is in an OCUnit harness. It’s not bad, it could be better.

So what are the problems with OCUnit?

  • repetition. Every time you write STAssert, you’re saying two things. Firstly, “hey, I’m using OCUnit”, which isn’t really useful information. Second, “what’s coming up is a test, read on to find out what kind of test”. Then you finally get to the end of the macro where you reveal what it is you’re going to do. This is the important information, but we bury it in the middle of the line behind some boilerplate.

    Imagine, instead, a hypothetical language where we could send messages to arbitrary expressions (ok that exists, but imagine it’s objc). Then you could do [[2+2 should] equal: 4]; which more closely reflects our intention.

  • repetition. In the same way that STAssert is boilerplate, so is sub classing SenTestCase and writing -(void)test at the beginning of every test method. It gives you no useful information, and hides the actual data about the test behind the boilerplate.

    Newer test frameworks in languages like C# and Java use the annotation features of those languages to take the fact that a method is a test out of its signature and make it metadata. ObjC doesn’t support annotations, so we can’t do that. But take a look at the way CATCH tests are marked up. You indicate that something is a test, and the fact that this means the framework needs to generate an objective-c++ class and call a method on it is encapsulated in the framework’s implementation.

  • repetition. You might think that there’s a theme developing here :-). If you write descriptive method names, you might have a test named something like -testTheNetworkConnectionIsCleanedUpWhenADownloadFails. Should that test fail, you’re told what is going wrong: the network connection is not cleaned up when a download fails.

    So what should you write in the mandatory message parameter all of the STAssert…() macros require? How about @"the network connection was not cleaned up when a download failed"? Not so useful.

  • organisation. I’ve already discussed how OCUnit makes you put tests into particular classes and name them in particular ways. What if you don’t want to do that? What if you want to define multiple groups of related tests in the same class, in the way BDD practitioners do to indicate they’re all part of the same story? What if you want to group some of the tests in one of those groups? You can’t do that.

I’m sure other people have other complaints about OCUnit, and that yet other people can find no fault with it. In this post I wanted to draw attention to the fact that there’s more than one way to crack a nut, and the vendor-supplied nutcracker is useful though basic.

On explaining stuff to people

An article that recently made the rounds, though it was written back in September, is called Apple’s Idioten Vektor. It’s a discussion of how the CCCrypt() function in Apple’s CommonCrypto library, when used in its default cipher block chaining mode, treats the IV (Initialization Vector) parameter as optional. If you don’t supply an IV, it provides its own IV of 0x0.

Professional Cocoa Application Security also covers CommonCrypto, CBC mode, and the Initialization Vector. Pages 79-88 discuss block encryption. The section includes sample code for both one-shot and staged use of the API. It explains how to set the IV using a random number generator, and why this should be done.[1] Mercifully when the author of the above blog post reviewed the code in my book section, he decided I was doing it correctly.

So both publications cover the same content. There’s a clear difference in presentation technique, though. I realise that the blog post is categorised as a “rant” by the author, and that I’m about to be the pot that calls the kettle black. However, I do not believe that the attitude taken in the post—I won’t describe it, you can read it—is constructive. Calling people out is not cool, helping them get things correct is. Laughing at the “fail” is not something that endears people to us, and let’s face it, security people could definitely be more endearing. We have a difficult challenge: we ask developers to do more work to bring their products to market, to spend more money on engineering (and often consultants), in return for potentially protecting some unquantified future lost revenue and customer hardship.

Yes there is a large technical component in doing that stuff, but solving the above challenge also depends very strongly on relationship management. Security experts need to demonstrate that we’re all on the same side; that we want to work with the rest of the software industry to help make better software. Again, a challenge arises: a lot of the help provided by security engineers comes in the form of pointing out mistakes. But we shouldn’t be self promoting douchebags about it. Perhaps we’re going about it wrong. I always strive to help the developers I work with by identifying and discussing the potential mistakes before they happen. Then there’s less friction: “we’re going to do this right” is a much more palatable story than “you did this wrong”.

On the other hand, the Idioten Vektor approach generated a load of discussion and coverage, while only a couple of thousand people ever read Professional Cocoa Application Security. So there’s clearly something in the sensationalist approach too. Perhaps it’s me that doesn’t get it.

[1]Note that the book was written while iPhone OS 3 was the current version, which is why the file protection options are not discussed. If I were covering the same topic today I would recommend eschewing CCCrypto for all but the most specialised of purposes, and would suggest setting an appropriate file protection level instead. The book also didn’t put encryption into the broader context of cryptographic protocols; a mistake I have since rectified.

Remember the future?

The future is notoriously hard to pin down. For example, what is Seattle’s lasting legacy from 20th Century technology? What would people have pointed to in, say, the 1970s? Of course, Seattle is the home of Boeing, who did a lot of construction for NASA (and bought most of the other companies that were also doing so) on projects like the Saturn V rocket and the Space Shuttle. Toward the end of the 1970s, those in the know in Seattle would have confidently claimed that the Shuttle’s weekly trips into space as the world’s longest-distance haulage provider will be central to 21st century Space Age technology. But neither the shuttle nor the Saturn V works any more, and nothing equivalent has come along to replace them (certainly not from Boeing). The permanent remnant of Seattle’s part in the space race comes from earlier on, when the USSR already had satellites in orbit, Gagarin had safely returned, and the USA wanted to assert its technological superiority over the Soviets. I’m talking, of course, about Seattle Center and its most famous landmark: a giant lift shaft with a restaurant at one end and a gift shop at the other.

Space NeedlePeople like to prognosticate about how our industry, society or civilization is going to change: both in the short term, and in the distant future (which can be anything beyond about three years in software terms). Sure, it’s fun to speculate. Earlier this year I took part in a panel with my arch-brother among others, where many of the questions were about what direction we thought Apple might take with the iPad, how existing companies would work their software to fit in post-PC devices, that sort of thing. That means that not only do we enjoy prognostication, but we seek it out. People enjoy playing the game of deciding what the future will be like, and hope for that spark of satisfaction of knowing either that they were right, or that they were there when someone else was right.

But why? It’s not as if we’re any good at it. The one thing that history lets us discover is that people who predict the future generally get it wrong. If they have the foresight to make really grandiose predictions they get away with it, because no-one finds out that they were talking out of their arses until well after they died. But just as the Space Needle has outlived the Space Shuttle, so the “next big thing” can easily turn out to be a fad while something apparently small and inconsequential now turns out to last and last.

Space NeedleOf course I’ll discuss the computing industry in this article, but don’t think this is specific to computing. In general, people go for two diametric visions of the future: either it’s entirely different from what came before, or it’s the same but a little better. The horses are faster, that kind of thing. Typically, experts in an industry are the people who find it hardest to predict that middle ground: a lot of things are the same, but one or two things have created a large change. Like the people at the air ministry who knew that superchargers were too heavy to ever allow Frank Whittle’s jet turbine to take off. Or the people who didn’t believe that people could travel at locomotive speeds. Or H.G. Wells, who predicted men on (well, in) the Moon, severely stratified society, life on other planets…just not the computer that was invented during his lifetime.

OK, so, computing. Remember the future of computers? The future will contain "maybe five" computers, according to Thomas Watson at IBM. I’m in a room now with about nine computers, not including the microcontrollers in my watch, hi-fi, cameras and so forth. There were around ten examples of Collosus produced in the 1940s. Why maybe five computers? Because computers are so damned heavy you need to reinforce the main frame of your floor to put them in. Because there are perhaps two dozen people in the world who understand computers. Because if you have too many then you have loads of dangerous mercury sloshing around. Because companies are putting themselves out of business attempting to sell these things for a million dollars when the parts cost nearly two million. And, finally, because there’s just not much you can do on a computer: not everyone needs ballistics tables (and most of the people who do want them, we don’t want to sell to).

Space NeedleEnough of the dim depths of computing. Let’s come into the future’s future, and ask whether you remember the other future of computers: the workstation. Of course, now we know that mainframes are old and busted, and while minicomputers based on transistor-to-transistor logic are cheaper, smaller, more reliable and all, they’re still kindof big. Of course, micros like the Altair and the Apple are clearly toys, designed as winter-evening hobbies for married men[*]. Wouldn’t it be better to use VLSI technology so that everyone can have their own time-sharing UNIX systems[**] on their desks, connected perhaps through the ultra-fast thinwire networks?

Better, maybe, but not best. Let’s look at some of the companies involved, in alphabetical order. Apollo? Acquired by HP. Digital? Acquired, circuitously, by HP. HP? Still going, but not making workstations (nor, apparently, much else) any more. IBM? See HP. NeXT? Acquired, making consumer electronics these days. Silicon Graphics? Acquired (after having left the workstation industry). Stanford University Networks? Acquired by a service company, very much in the vein of IBM or HP. Symbolics, the owners of the first ever .com domain? Went bankrupt.

The problem with high-end, you see, is that it has a tendency to become low-end. Anything a 1980s workstation can do could be done in a “personal” computer or a micro by, well, by the 1980s. It’s hard to sell bog standard features at a premium price, and by the time PCs had caught up to workstations, workstations hadn’t done anything new. Well, nothing worth talking about…who’d want a camera on their computer? Notice that the companies that did stay around-IBM and HP-did so by getting out of the workstation business: something SGI and Sun both also tried to do and failed. The erosion of the workstation market by the domestic computer is writ most large in the Apple-NeXT purchase.

Space NeedleSo workstations aren’t the future. How about the future of user interfaces? We all know the problem, of course: novice computer users are confused and dissuaded by the “computery-ness” of computers, and by the abstract nature of the few metaphors that do exist (how many of you wallpaper your desktop?). The solution is obvious: we need to dial up the use of metaphor and skeuomorphism to make the user more comfortable in their digital surroundings. In other words, we need Bob. By taking more metaphors from the real world, we provide a familiar environment for users who can rely on what they already know about inboxes, bookshelves, desk drawers and curtains(!) in order to navigate the computer.

Actually, what we need is to get rid of every single mode in the computer’s interface. This is, perhaps, a less well-known future of computing than the Bob future of computing, despite being documented in the classic book The Humane Interface, by Jef Raskin. The theory goes like this: we’ve got experience of modal user interfaces, and we know that they suck. They force the user to stop working while the computer asks some asinine question, or tells them something pointless about the state of their application. They effectively reverse the master-slave relationship, making the user submit to the computer’s will for a while. That means that in the future, computers will surely dispose of modes completely. Well, full modes: of course partial modes that are entirely under the user’s control (the Shift key represents a partial mode, as does the Spotlight search field) are still permitted. So when the future comes to invent the smartphone, there’ll be no need for a modal view controller in the phone’s API because future UI designers will be enlightened regarding the evils of modality.

A little closer to home, and a little nerdier, do you remember the future of the filesystem? HFS+ is, as we know, completely unsuitable as a filesystem for 2009 so future Macs will instead use Sun’s ZFS. This will allow logical volume management, versioned files…the sorts of goodies that can’t be done on HFS+. Oh, wait.

These are all microcosmic examples of how the future of computing hasn’t quite gone according to the predictions. I could quote more (one I’ve used before is Bob Cringely’s assertion in 1992 that in fifteen years, we’ll have post-PC PCs; well I’m still using a PC to write this post and it’s 2011), but it’s time to look at the bigger picture, so I’m going to examine why the predictions from one particular book have or haven’t come about. I’m not picking this book because I want to hate on it; in fact in a number of areas the predictions are spot on. I’m picking on this book because the author specifically set out to make short, medium and long-term forecasts about the silicon revolution, and the longest-term predictions were due to have become real by the year 2000. The book is The Mighty Micro: Impact of the Computer Revolution by Dr. Christopher Evans, published in 1979.

According to the Mighty Micro the following should have all happened by now.

  • Openness and availability of information leads to the collapse of the Soviet Union. ✓
  • A twenty-hour working week and retirement at fifty. ✗
  • Microcontroller-based home security. ✓ For everyone, replacing the physical lock-and-key. ✗
  • Cars that anticipate and react to danger. ✓ As the standard. ✗
  • A “wristwatch” that monitors pulse and blood pressure. ✓
  • An entire library stored in the volume of a paperback book. ✓
  • A complete end to paper money. ✗
  • An end to domestic crime. ✗

So what happened? Well, “processors and storage will get smaller and cheaper” was the prevailing trend from the forties to the seventies, i.e. over the entire history of electronic computing. Assuming that would continue, and that new applications for tiny computers would be discovered, was a fairly safe bet, and one that played out well. The fundamental failures behind all of the other predictions were twofold: that such applications would necessarily replace, rather than augment, whatever it was that we were doing before computers, and that we would not find novel things to do with our time once computers were doing the things we already did. The idea was that once computers were doing half of our work, we would have 50% as much work to do: not that we would be able to do other types of work for that 50% of our working week.

One obvious thing we-well, some of us-have to do now that we didn’t before is program computers. Borrowing some figures from the BSA, there were 1.7M people working in software in the US in 2007, earning significantly more than the national average wage (though remember that this was during the outsourcing craze, so a lot of costs and jobs even for American companies might be missing here). The total worldwide expenditure on (packaged, not bespoke) software was estimated at $300bn. Once you include the service aspects and bespoke or in-house development, it’s likely that software was already a trillion-dollar industry by 2007. Before, if you remember, the smartphone app gold rush.

This is a huge (and, if we’re being brutally honest, inefficient) industry, with notoriously short deadlines, long working hours, capricious investors and variable margins. Why was it not predicted that, just as farmhands became machine operators, machine operators would become computer programmers? That the work of not having a computer would be replaced by the work of having a computer?

So, to conclude, I’ll return to a point from the article’s introduction: that making predictions is easy and fun, but making accurate predictions is hard. When a pundit tells you that something is a damp squib or a game-changer, they might be correct…but you might want to hedge your bets. Of course, carry on prognosticating and asking me to do so: it’s enjoyable.

[*] This is one of the more common themes of futurology; whatever the technological changes, whatever their impacts on the political or economic structure of the world, you can bet that socially things don’t change much, at least in the eye of the prognosticators. Take the example of the Honeywell Kitchen Computer: computers will revolutionise the way we do everything, but don’t expect women to use them for work.

[**] Wait, if we’ve each got our own computer, why do they have to be time-sharing?

On comment docs

Something I’m looking at right now is generation of (in my case, HTML) API documentation from some simple markup format. The usual way to do this is by writing documentation markup inline in the source code, using specially formatted comments in header files.

The point

Some people argue that well-written source code should be its own documentation. Well, that’s true, it is: but it’s documentation with limited utility. Source code provides the following documentation:

  • Document, for the compiler’s benefit, the machine instructions that the compiler should generate
  • Document, for the programmer’s benefit, the machine instructions that the compiler should generate

Developing a high-level model of how software works from its source code is possible, but mentally taxing. It’s not designed for that. It would be like asking an ant to map the coastline of Africa: it can be done, but the information available is at entirely the wrong scale.

Several other approaches for high-level documentation of software systems exist. Of course, each of them is not actually the source code, but a model: a map of Africa is not actually Africa, but if you want to know what the coastline of Africa looks like then the map is a very useful model.

Comment documentation is one such model of software. Well-written comment docs explain why you might use a method or class, and how its properties or parameters help you to use it. It gives you a sense of how the classes fit together, and how you can exploit them. They’re called comment docs because they go into comments right alongside the source they document, usually marked up with particular tokens. As an example of a token, many documentation comment systems let me use a line like this:

@author Graham J. Lee

to indicate that I wrote a particular part of the project.

Of course, this documentation could go anywhere, so why put it into the header files? For a start, you already have the header files, so you’re not having to maintain two parallel hierarchies of content. Also, the proximity of the documentation to the source code means that there’s a higher probability (still not unity, but higher) that a developer who changes the intent or usage of a method will remember to update the documentation. Additionally, it means that the documentation can be no more verbose than required: anything that is obvious from the source (the names of methods and their parameters, for example) can be discovered from the source. This is not inconsistent with my earlier statement about source-as-documentation: you can easily find out a method signature without having to grub through the actual program instructions.

Finally, it means that when you’re working with the source, you have the documentation right there. This is one very common way to interact with comment documentation. The other way is to use a tool to create some friendly formatted output (for me, the goal is HTML) by reading the source files and extracting the useful information from the comments.

Doxygen

I have for the last forever (alright, three years) used Doxygen, for a couple of reasons:

  • The other people on my team at the time I adopted it were already using it
  • Since then, it has gained support for generating Xcode documentation sets, which can be viewed inside the Xcode organizer.

It’s not great, though. It’s a very complicated tool that tries to do all things for all people, so the configuration is huge and needs a lot of consideration. You can customise the look of the output using a CSS file but you pretty much need to as the default output looks like arse. Making it actually output different stuff is trickier, as it’s a C++ project and I’ve only ever learned enough C++ to customise Clang.

HeaderDoc

So I decided to fish around for alternatives. Of course, we know that Apple uses (and ships) HeaderDoc, but it uses its own comment format. Luckily, its comment format is identical to Javadoc, which is the format used by Doxygen. You can get headerdoc2html to look for the /** trigger by passing the -j flag.

Speaking of headerdoc2html, this is one of two programs in the HeaderDoc distribution. The other is gatherheaderdoc, which generates a table of contents from a collection of output files. Each of these is a well-written and well-documented perl script, which makes extension and modification super-easy.

Neither should be immediately required, in fact. The tool understands all of the Objective-C language features, and some extra bonus things like availability macros and groups of related methods. The default output basically looks like someone forgot to apply the style sheet to developer.apple.com. It’s trivial to configure HeaderDoc to use an external style sheet, and you can even create a custom template HTML file so that things appear in whatever fashion you want. Another useful configuration point is the C preprocessor, which you can get HeaderDoc to run through before interpreting the documentation.

Autogsdoc

The GNUstep project uses its own comment documentation tool called Autogsdoc. The fact that the autogsdoc documentation has buggy HTML does not fill one with confidence.

In fact, Autogsdoc’s output is unstyled XHTML 1.0, so it would be easy to style it to look more useful. Some projects that I’ve seen appear to have frames-based HTML, which is unfortunate.

Autogsdoc uses inline comment documentation, the same as the other options we’ve looked at. However, its markup is significantly different, as it uses SGML-style tags inside the documentation. It has a (well-documented, of course) Objective-C implementation with a good split of responsibilities between different classes. Hacking about on its innards shouldn’t be too hard for any motivated Cocoa coder.

Footnote

I know this has been annoying you all since the first paragraph in the section on HeaderDoc: */.

On Timeless Programming Books

Recently, the Dog Spanner wrote about Programming With Quartz, a book written at the tail end of 2005 but which is still useful to Mac developers everywhere. I have to agree, this book is still on my shelf and gets an airing every now and then when I need to do battle with custom drawing (even on iOS).

Today, I had a related moment of epiphany related to the immortal nature of a book, as I had a problem with memory allocation that saw me reaching to Mac OS X Internals: a Systems Approach by Amit Singh. This book was released just before Apple’s transition to Intel hardware, but is still for me a definitive reference on how Mac OS X works. If I need to know about the innards of HFS+, the memory allocator or anything at a similar level, it is to this book I turn.

There’s never really been anything that investigates the higher level components of Mac OS X in quite the same depth. When I need to refresh my knowledge of the UNIX APIs, I turn first to Advanced UNIX Programming, a book first published in 1985(!); but it is the 2004 edition that is still a canonical description of programming in the UNIX environment.

Notice that each of these books is around five years old, and yet still I find myself referring to them frequently. In the fast-changing world of software development, where books on the iPhone 3 SDK are woefully outdated, that’s a great achievement.

Honourable mention: a book I have on my shelf that deserves discussion here is Object Oriented Programming: An Evolutionary Approach by Brad Cox. Written in 1986, this lays out his vision for component-based software development using object oriented programming languages. In it he describes a little language he created called Objective-C and uses it to explain his vision. This book demonstrates that not all computing principles are long-lived. Today’s programming practices look nothing like the “Software ICs” of OOP, although we’re still stuck using Objective-C.