Let’s talk about self-documenting code

You think your code is self-documenting. That it doesn’t need comments or Doxygen or little diagrams, because it’s clear from the code what it does.

I do not think that that is true.

Even if your reader has at least as much knowledge of the programming language you’ve used as you have, and at least as much knowledge of the libraries you’ve used as you have, there is still no way that your code is self-documenting.

How long have you been doing your job? How long have you been talking to experts in the problem domain, solving similar problems, creating software in this region? The likelihood is, whoever you are, that the new person on your team has never done that, and that your code contains all of the jargon terms and assumptions that go with however-much-experience-you-have experience at solving those problems.

How long were you working on that story, or fixing that bug? How long have you spent researching that specific change that you made? However long it is, everybody else on your team has not spent that long. You are the world expert at that chunk of code, and it’s self-documenting to you as the world expert. But not to anybody else.

We were told about “working software over comprehensive documentation”, and that’s true, but nobody said anything about avoiding sufficient documentation. And nobody else has invested the time to understand the code that you just wrote that you did, so the only person for whom your code is self-documenting is you.

Help us other programmer folks out, think about us when avoiding documentation.

It’s about the thinking

At some point in the past, programmers used to recommend drawing flowcharts before you start coding. Then they recommended creating CRC cards, or acting through how the turtle will behave, or writing failing tests, or getting the types to match up, or designing contracts, or writing proofs, but the point is that in each case they’re there for eliciting thought before the code gets laid down.

None of these things is mutually exclusive, none of these things is the one true way, but the fact that they all isolate some part of solving the problem from some part of coding the solution is the telling point. The problem is not having the correct type system or test coverage or diagram format, the problem is trying to work in two (or more) levels of abstraction – the problem domain and the computer – at the same time.

Head of Architecture

My current job title is Head of Architecture, though the word “architecture” means different things to different people in the world of software. So what does it mean to me, what do I do when I’m playing Head of Architecture?

I follow Perry and Wolf in Foundations for the Study of Software Architecture by drawing the analogy between software architecture and built environment architecture, not network or electronics architecture. The work of a built environment architect, particularly one who follows the path laid out by Christopher Alexander, combines elements, form and aesthetics by creating a system that complements and enhances its environment. Software systems are deployed into environments (with existing people, processes, cultural norms) and developed in environments, and should make those environments better for the people who are interacting with them, while also meeting the functionality, performance, security and other goals of the system.

But that doesn’t actually explain what I do, which is more about letting other people do things that are “architecture” than about “doing architecture” for them. Programmers, ops folks, QA people, product owners, and others frequently make decisions that have wide (and hence “architectural”) impact, and I think it’s better to enable that and follow up by asking how it impacts the rest of the system, to refine the choices made, than to stop people from making those decisions for them in the name of “being the architect”.

So playing software architect for me tends to be more about creating a forum in which people can present aspects of the problems they’re trying to solve, needing to solve soon, or the solutions they’re exploring, and getting a view from multiple teams and multiple functions about those problems and solutions. Making sure that ops know what devs are doing, that product team Alpha knows how product team Aleph are solving that issue, and so on.

How about the stereotype that software architects program using Visio or Powerpoint? In my case, I program using JavaScript. I do make documentation, to make sure that decisions made in the forum are captured, that proposed or current approaches can be seen and understood. And yes, much of that documentation is diagrammatic. But ultimately I’m a programmer on a software team too, and that documentation has to reflect working, valuable software. That is, while there is value in comprehensive documentation, we value working software more.

Bottom-up teaching

We’re told that the core idea in computer programming is problem-solving. That one of the benefits of learning about computer programming (one that is not universally accepted) is gaining the skill of problem decomposition.

If you look at real teaching of computing, it seems to have more to do with solution composition than problem decomposition. The latter seems to be background noise: here are the things you can build solutions with, presumably at some point you’ll come across a solution that’s the same size and shape as one of your problem components though how is left up to you.

I have many books on programming languages. Each lists the features of the language, and gives minimally complex examples of the use of those features. In that sense, Kernighan and Ritchie’s “The C Programming Language” (section 1.3, the for statement) is as little an instructional in solving problems using a computer as Eric Nikitin’s “Into the Realm of Oberon” (section 7.1, the FOR loop) or Dave Thomas’s “Programming Elixir” (section 7.2, Using Head and Tail to Process a List).

A course textbook on bitcoin and blockchain (Narayanan, Bonneau, Felten, Miller and Goldfeder, “Bitcoin and Cryptocurrency Technologies”) starts with Section 1.1, “Cryptographic hash functions”, and builds a cryptocurrency out of them, leaving motivational questions about politics and regulation to Chapter 7.

This strategy is by no means universal: Liskov and Guttag’s “Program Development in Java” starts out by describing abstraction, then looks at techniques for designing abstractions in Java. Adele Goldberg and Alan Kay described teaching Smalltalk by proposing exploratory projects, designing the objects that model the problem under consideration and the way in which they will communicate, then incrementally filling in by designing classes and methods that have the desired properties. C.J. Date’s “An Introduction to Database Systems” answers the question “why databases?” before introducing the relational model, and doesn’t introduce SQL until it can be situated in the context of the relational model.

Both of these approaches, and their associated techniques (the bottom-up approach and solution construction; the top-down approach and problem decomposition) are useful; the former leads to progress and the latter leads to understanding. But both must be taken in concert, because understanding without progress leads to the frustration of an unsolved problem and progress without understanding is merely the illusion of progress.

My guess is that more programmers – indeed whole movements, when we consider the collective state of things like OOP, functional programming, BDD, or agile practices – are in the “bottom-up only” group than in the “top-down only” or “a bit of both” groups. That plenty more copies of Introduction to Programming in [This Week’s Hot Language] have been sold than Techniques for Making Your Problem Amenable to Computation. That the majority of software really does comprise of solutions looking for problems.

Choose boring employers

Amusingly, my previous post choose boring employees was shared to hacker news under the off-by-one erroneous title choose boring employers. That seemed funny enough to run with, but what does it mean to choose boring employers?

One interpretation is that a boring employer is one where you do not live in interesting times. Where you can get on with your job, and with finding new and better ways to do your job, without constantly fighting fires.

But what if you’re happiest in an environment where you are fighting fires? In that case, you probably should surround yourself with arsonists.

Another interpretation is to invert the discussion in Choose Boring Employees: find an employer who spends their innovation tokens wisely. One who’s OK with the answer to “how do I store these tuples of known structure” being “in a relational database”, or one who doesn’t mind when the answer to “what platform should we base our whole business on” starting with “I skim-read a blog post on HN when I was riding MUNI this morning and…”.

But, let’s be clear, there’s a place for the shiny new technology. Sometimes you do need to spend your innovation tokens, so you don’t want to be somewhere that won’t let you do it at all. Working on a proof of concept, you want to get to proof quickly, so it may be time to throw caution to the wind (unless the concept you’re trying to prove involves working within some cautious boundaries). So boring need not get as far as frustrating.

Choose boring employees

An idea I’ve heard from many directions recently is that “we” (whoever they are) “need to be on the latest tech stack in order to attract developers”. And yes, you do attract developers that way. Developers who want to be paid to work on the latest technology.

Next year, your company will be a year more mature. Your product will be a year more developed. You will have a year more customers. You’ll have a year more tech debt to pay off.

And your cutting-edge tech stack will be so last year. Your employees will be looking at the new startup in the office next door, and how they’re hiring to work on the latest stack while you’re still on your 2017 legacy technology.

In which GNUstep confuses and ultimately disappoints

I’m not the most hardcore of GNUstep people, but I’m certainly somewhat invested. I’ve been building apps, lurking in lists, and contributing code on and off for around 13 years, including a job working with a few of the maintainers. I am trying to build the corpus of documentation intrinsic to GNUstep, i.e. that which doesn’t require interpretating Apple’s Cocoa docs through a compatibility lens.

So when I discovered that an app I need doesn’t exist and that I’d have to make it myself, GNUstep was the obvious choice. I could take all that experience and investment in Objective-C and Cocoa and build the app I need. I’d probably be able to contribute to the framework along the way, too: a previous side project of mine resulted in adding NSUUID to gnustep-base.

This time, however, the amount of framework support needed proved too great. The thing I want is an app to help me organise and read some PDFs, and there’s no PDF-reading support in GNUstep. Well, let me be more specific: there has been, but it’s disappeared. There was a library called PopplerKit that wraps the poppler PDF-rendering library. However the only references to that are to a subversion repo in the now-defunct Gna! project: i.e. there is no PopplerKit available any more.

One option I considered is to write my own poppler wrapper, perhaps exposing the same interface as Apple’s PDFKit. This would probably be welcome in the GNUstep community, would improve their libraries’ API compatibility with Cocoa, and be useful to other developers. However, it’s also a distraction from what I’m trying to do, which is to make my PDF app.

So I decided to choose a different technology, where PDF rendering is already available. Poppler has a Qt5 interface already, so I built my app using C++ and Qt5. After half an hour of effort I already have a page-turning PDF viewer, which is not pleasant but enough to let me be confident that I can build the rest of the application.

I had thought that this would be the end of the story. I would have a moral here (still a useful moral) about picking the technology that lets you get your task solved, being realistic about how much you’re drawn to your pet project or favourite “stack” and whether that bias is worth tipping the balance: in many cases, it probably is, but be aware of what it’s costing you. I expected to talk about code rot, and to despair over the fact that a library I knew about and was aware could solve my problem no longer existed.

Then, in preparing this post, I happened to dive through an unrelated repository on github, the GNUstep Applications Project (a repository containing a collection of unrelated applications), and discovered that it contains a PDFKit implementation. It doesn’t contain PDFView so can’t be used as a reader, so it doesn’t change the choice over which of GNUstep or Qt should be used now. But it does change the amount of effort required to reimplement PDFKit on top of GNUstep, should I revisit that in the future.

So there’s a point in here about discoverability too. I did quite a lot of searching to find that there isn’t a PopplerKit, and also found that there wasn’t a GNUstep PDFKit. Even so, there was, and even more searching turned it up. En route I found that a different technology had what I wanted, and changed my choice of programming language and application framework in order to use that technology in my project. You can’t merely build it and expect them to come, you have to show them that it’s there too.

Your build needs to be better

I’ve said it before, build systems are a huge annoyance. If your build is anything other than seemingly instantaneous, it’s costing you severe money.

Your developers are probably off reading HN, or writing blog posts about how slow builds cost them, while the build is going. When they finish doing that, which may be some time after the build completes, they’ll have forgotten some of what they were doing and need to spend some time getting back up to speed.

Your developers are probably suspicious of any build failure, thinking that “the build is flaky” rather than “I made a mistake”. They’ll press the button again and go back to HN. When the same error occurs twice, they might look into it.

Your developers probably know that the build is slow, but not which bit of the build is slow. And they don’t have time to investigate that, where it takes so long to get any work done anyway. So everyone will agree that “there is a problem”, but nothing will get done. Or maybe cargo-cult things will get done, things that speed up “builds” but are not the problem with your build.

The Joel test asks whether you can make a build in one test. Insufficient. If you notice when you’re making a build, you’re slowing your developers down.

…which is not always the worst thing, of course. Sometimes a lengthy translation step from some source language to some optimised form of a machine language program yields better results for your customers, because they get to use a faster program and don’t need to care about the time taken to prepare that program. But let’s be clear: that’s part of the release, and your developers don’t always need to be working from the released product (in fact, they’re usually not). Releases should be asynchronous, and the latency between having something ready to be released and having released it can be fairly high, compared with the latency between having created some source and being able to investigate its utility.

Nonetheless, that should all go off in the background. So really, builds and releases should both be non-events to the developers.

The package management paradox

There was no need to build a package management system since CPAN, and yet npm is the best.
Wait, what?

Every time a new programming language or framework is released, people seem to decide that:

  1. It needs its own package manager.

  2. Simple algorithms need to be rewritten from scratch in “pure” $language/framework and distributed as packages in this package manager.

This is not actually true. Many programming languages – particularly many of the trendy ones – have a way to call C functions, and a way to expose their own routines as C functions. Even C++ has this feature. This means that you don’t need any new packaging system, if you can deploy packages that expose C functions (whatever the implementation language) then you can use existing code, and you don’t need to rewrite everything.

So there hasn’t been a need for a packaging system since at least CPAN, maybe earlier.

On the other hand, npm is the best packaging system ever because people actually consume existing code with it. It’s huge, there are tons of libraries, and so people actually think about whether this thing they’re doing needs new code or the adoption of existing code. It’s the realisation of the OO dream, in which folks like Brad Cox said we’d have data sheets of available components and we’d pull the components we need and bind them together in our applications.

Developers who use npm are just gluing components together into applications, and that’s great for software.