Skip to content

The Principled Programmer

[Note: this post represents the notes made for my talk at iOS Dev UK 2014. As far as I’m aware, the talk isn’t available on the tubes.]

The Principled Programmer

The first thing to be aware of is that this post is not about my
principles. It’s sort-of about your principles, in a way.

On dichotomies

Let’s look at two games. You may not have heard of Chaturanga (unless
you practice yoga, but I’m talking about a different Chaturanga), but
it’s the ancient game that eventually evolved into Chess.

You may not have heard of Nard either, but it’s a very different game
that grew up into Backgammon. There’s a creation myth surrounding
these two games, that says they were invented at the same time. Some
leader thousands of years ago wanted two games; one a game of skill
and the other a game of chance.

The thing is, you can lose at chess by chance: if you happen to be
having an off day and miss a key move that you’d often
make. Similarly, you can lose at backgammon through lack of skill: by
choosing to move the wrong pieces.

We were presented with two options: skill (chaturanga) and not-skill
(nard). However, the games do not actually represent pure states of
the two concepts; they’re more like a quantum system where the
real-world states can be superpositions of the mathematically “pure”

All of this means that we can’t ignore states in-between the two
poles. Such ignorance has a name in the world of critical analysis:
the fallacy of the excluded

That is not the situation we have in bivalent logic, including the
mathematical Boolean formulation frequently used to model what’s going
on in computers. This has the law of the excluded
, which
says that a proposition must either be true or false.

In this case, the fact is that the two propositions (you are playing a
game of skill, or you are playing a game of chance) do not exactly
match the two real possibilities (you are playing chaturanga, or you
are playing nard). There’s a continuum possibility (you are using some
skill and some chance), but a false dichotomy is proposed by the
presentation in terms of the games.

On rules

The existence of a rule allows us to form a bivalent predicate: your
action is consistent with the rule. That statement can either be true
or false, and the middle is excluded.

This means we have the possibility for the same confusion that we had
with the games: compliance with the rule may be bivalent, but what’s
going on in reality is more complicated. We might accidentally exclude
the middle when it actually contains something useful. Obviously that
useful thing would not be in compliance with the rule. So you can
think about a rule like this: a statement is a rule when you can
imagine contraventions of the statement that are of no different value
than observances of the statement. Style guides are like this: you can
imagine a position that contravenes the rules of your style guide
that is of no lesser or greater value: following another style

Of course, the value of a style guide comes not from the choice of
style, but from the consistency derived from always adhering to the
rule. It doesn’t matter whether you drive on the left or the right of
the road, as long as everybody chooses the same side.

One famous collection of rules in software
engineering is Extreme Programming. Kent Beck described hearing or
reading about various things that were supposed to be good ideas in
programming, so he turned them up to eleven to see what would
happen. Here are some of the rules.

  • User stories are written. It’s easy to imagine (or recall)
    situations in which we write software without writing user stories:
    perhaps where we have formal specifications, or tacit understandings
    of the requirements. So that’s definitely a rule.

  • All production code is pair programmed. The converse – not all
    production code is pair programmed – poses no problem. We can imagine that the two conditions are different, and that we might want to choose one over another.

Rules serve two useful functions, of which I shall introduce one
now. They’re great for beginners, who can use them to build a scaffold
in which to place their small-scale, disjoint bits of knowledge. Those
bits of knowledge do not yet support each other, but they do not need
to as the rules tell us what we need to apply in each situation.

The software engineering platypus

Having realised that our rules are only letting us see small pieces of
the picture, we try to scale them up to cover wider
situations. There’s not really any problem with doing that. But we can
get into trouble if we take it too far, because we can come up with
rules that are impossible to violate.

A platitude, then, is a statement so broad that its converse cannot be
contemplated, or is absurd. Where a rule can be violated without
hardship, a platitude cannot be violated at all – or at least not

The problem with platitudes is that because we cannot violate them,
they can excuse any practice. “I write clean code”: OK, but I don’t
believe I know anybody who deliberately writes dirty code. “This
decision was pragmatic”: does that mean any other option would be
dogmatic? But isn’t “always be pragmatic” itself dogma?

Platitudes can easily sweep through a community because it’s
impossible to argue against them. So we have the software
craftsmanship manifesto, which values:

  • A Community of Professionals. As any interaction
    between people who get paid comes under this banner, it’s hard to see
    what novelty is supplied here.

  • Well-Crafted Software. Volunteers please for making shitty

The Principled Programmer.

There must be some happy medium, some realm in which the statements we
make are wider in scope, and thus more complex, than rules, but not so
broad that they become meaningless platitudes that justify whatever
we’re doing but fail to guide us to what we should be doing.

I define this as the domain of the principle, and identify a principle
thus: a statement which can be violated, where the possibilities of
violation give us pause for thought and make us wonder about what it
is we value. To contrast this with the statements presented earlier:

  • violate a rule: meh, that’s OK, the other options are just as good.

  • violate a platitude: no, that’s impossible, or ludicrous.

  • violate a principle: ooh, that’s interesting.

Coming up with good principles is hard. The principles behind the
agile manifesto contain some legitimate principles:

  • Our highest priority is to satisfy the customer through early and
    continuous delivery of valuable software. Interesting. I can imagine
    that being one of many priorities of which others might be higher:
    growing the customer base, improving software quality, supporting what
    they’re using now and deferring delivery of new software until it’s
    needed. I’ll have to think about that.

  • Working software is the primary measure of
    progress. Interesting. This seems to suggest that paying off technical
    debt – exchanging one amount of working software for another amount of
    working software over a period of time – is not progress. I’ll have to
    think about that.

But then it also contains rules:

  • Deliver working software frequently, from a couple of weeks to a
    couple of months, with a preference to the shorter timescale. We ship
    a couple of times a day, and I don’t feel that’s worse.

  • Business people and
    developers must work together daily throughout the project. Is there
    anything wrong with every other day?

And platitudes:

  • Build projects around motivated individuals. Give them the
    environment and support they need, and trust them to get the job
    done. I cannot imagine a situation where I would hire people who do
    not want to do the work.

  • Continuous attention to technical excellence and good
    design enhances agility. This is tautological, as technical excellence
    and good design can be defined as those things that enable our goals
    and processes.

Why is it hard? I believe it’s because it’s highly personal, because
what you’re willing to think about and likely to get benefit from
thinking about depends on your own experiences and interests. Indeed
I’m not sure whether I want to define the principle as I have done
above, or whether it’s the questions you ask while thinking about
the validity of those things that are really your principles.

## Nice principles. Now go and turn them into

The thing about thinking is that I don’t want to do it when I don’t
need to. My currency is thought, so if I’m still thinking next year
about the things I was considering this year, I’m doing it wrong.

Principles are great for the things that need to challenge
the way we work now. But they should be short-lived. Remember the
beginner use of rules was only one of two important contexts? The
other context is in freeing up cognitive space for people who
previously had principles, and now want to move on to have new
principles. In short-circuiting the complex considerations you
previously had, mentally automating them to prepare yourself for
higher-level considerations.

Notice that this means that it’s a rule in isolation that doesn’t cause us any problems to violate. It may be that the rule was derived from a principle, so some thought went into its construction. Without that information, all we can see is that there are two possibilities and we’re being told that one of them is acceptable.

The challenge that remains is in communication, because it doesn’t
help for the context of a rule to be misidentified. If you’re a
beginner, and you describe your beginner rule and someone takes it as
an expert rule, they might end up talking about perspectives that
you’re not expecting. Also if you’re an expert and your expert rule is
perceived as a beginner rule, you might end up having to discuss
issues you’ve already considered and resolved.

So by all means, identify your principles. Then leave them behind and
discover new ones.

Selectively caring

When Choose Boring Technology was published earlier this year, it hit home for me. If you’re spending money trying to ship, say, a music notation app, there’s no point in rewriting the operating system scheduler.

Let’s say every company gets about three innovation tokens. You can spend these however you want, but the supply is fixed for a long while. You might get a few more after you achieve a certain level of stability and maturity, but the general tendency is to overestimate the contents of your wallet. Clearly this model is approximate, but I think it helps.

If you choose to write your website in NodeJS, you just spent one of your innovation tokens. If you choose to use MongoDB, you just spent one of your innovation tokens. If you choose to use service discovery tech that’s existed for a year or less, you just spent one of your innovation tokens. If you choose to write your own database, oh god, you’re in trouble.

Today the personal lesson finally percolated out of this team decision. Why is it I can’t make any progress on software? I care about too many things. This isn’t just about choosing technology, which is a small part of what technologists do (though it does imply passing all of the other caring, about ethics and finding potential customers and selecting customers and retaining customers and making money and hiring people and firing people and evaluating performance and making the lunch and designing a product and creating an experience and plumbing the toilets and other things to other people).

It’s about those things, the technical things, and more. If I care about all of those things then there’ll always be something not quite right about what I’m doing, which will be morale-destroying. Or it will make me work on fixing the thing that isn’t quite right, rather than whatever I need to make progress on. Or both. Often both. And, of course, given the size of our industry and the number of bloviators in it, for any subject upon which it’s possible to have an opinion there will be multiple opinions written, so plenty of possibilities to research how to care about whatever it is I’ve decided to care about.

Choosing what to care about is a skill, and a valuable one to have. Choosing not to care about something doesn’t necessarily mean ignoring its impact, though that is one way to deal with it. You can hire someone else (directly or under contract) to care about it, make an arbitrary choice and go with that, or get someone else to make an arbitrary choice that you then follow.

If the division of labour was the driver of efficiency in the industrial age, then the division of caring is that driver in the information age. Now to choose what to care about.

Week Seven

Having spent a few weeks trying all of the things and letting life happen, this week was about selection and focus. What should I actually concentrate on, and put energy into?

It’s time to add some structure to this situation. Dropping all of the things and taking life as it comes was relaxing, enjoyable, and necessary. Re-moulding things and building something new out of the parts will be necessary to provide a new sense of engagement and purpose.

Week six

This is a post about marmalade.

A “life hack”, if you will, that I learnt from Simon Stewart is to find things that I don’t do because I don’t like them, or think I don’t like them, and to do them. Do them again. Find out whether there’s anything to like about them, whether I missed something important, or whether my tastes have simply changed.

That is why breakfast this weekend featured the phrase “could you pass the marmalade, please, I don’t like it”.

This is not a post about marmalade.

Week five

“You look so much happier!”

I get the best compliments. Also, I feel so much happier. I have put people, friendships, connections, and experiences first, and am taking advantage of the rewards.

One such experience was a visit a couple of further education (16-18 years old) computing classes in my county. I was mostly there to talk about my background in the industry and help them to understand what jobs there are and what employers look for. I was blown away by the things these students could make in their robotics classes though, with Mindstorms kits talking to laptops and mobile apps.

That was a huge step up from the turtle robots we had shuffling around the floor. in my school. And it still feels like there’s even more opportunity there, like there’s a huge gap between what a student sees computers being capable of and what they can do with the tools we (the industry) give them.

Week four

Pragma conf was a lot of fun! I met loads of new and old friends, which led to meaningful conversations about what we do, what we sometimes feel we should do, and what we want to do.

One such conversation, with Chris Eidhof, was about how we think about programming. It was brought to mind again recently by Derek Jones’s post on Lisp and functional programming languages. His description of the Lisp community reminds me a lot of my own post on the tankard brigade, in that what keeps Lisp going is the exclusivity, and the need to know arcane rules to get things done when non-purists have much easier means to the same end.

What intrigues me about that is that it puts me into the tankard brigade, which gives me a lot to reflect on. When I look at Lisp I see a parsimony, a lack of arbitrary rules. It looks like there’s a single underlying metaphor, and everything in my solution can be expressed through that metaphor (no matter how I solved the problem). I look at Lisp and see functions. I look at Io or Self and see messages. But when I look at supposedly more accessible languages I see a bag of edge cases and special “oh, you want to do that? No, you need one of these” conditions, with no metaphor.

Staying power

You would imagine that by now I would have come to realise how long my attention span is and worked to find projects that fit within it, but no. This is one of the changes I need to make soon.

So often I start a project really excited by it, but am really excited by something else before the end. Book projects always work that way, and quite a few software projects. Sometimes even talks, given a long enough lead time between being asked for a topic and actually giving the talk.

The usual result is that I become distracted before the end of the project, which leads to procrastination. That then makes it take longer, which only increases the distraction and disengagement.

What I’m saying is that if I ever say that I’m thinking of starting a PhD, you have my permission to chastise me. Four years is not within my observed boredom limit. Six months is closer to the mark.

*-Oriented Programming

Much is written about various paradigms or orientations of programming: Object- (nee Message-) Oriented, Functional, Structured, Dataflow, Logic, and probably others. These are often presented as camps or tribes with which to identify. A Smalltalk programmer will tell you that they are an Object-Oriented programmer, and furthermore those Johnny-come-latelies with their Java are certainly not members of the same group. A Haskell programmer will tell you that they are a functional programmer, and that that is the only way to make working software (though look closely; their Haskell is running on top of a large body of successful, imperative C code).

Notice the identification of paradigms with individual programming languages. Can you really not be object-oriented if you use F#, or is structured programming off-limits to an Objective-C coder? Why is the way that I think so tightly coupled to the tool that I choose to express my thought?

Of course, tools are important, and they do have a bearing on the way we think. But that’s at a fairly low, mechanical level, and programming is supposed to be about abstraction and high-level problem solving. You’re familiar with artists working with particular tools: there are watercolour painters and there are oil painters (and there are others too). Now imagine if the watercolour painting community (there is, of course, no such thing) decreed that it’s impossible to represent a landscape using oil paints and the oil painting community declared that watercolours are “the wrong tools for the job” of painting portraits.

This makes no sense. Oil paints and watercolour paints define how the paint interacts with the canvas, the brush, and the paint that’s already been applied. They don’t affect how the painter sees their subject, or thinks about the shapes involved and the interaction of light, shadow, reflection, and colour. They affect the presentation of those thoughts, but that’s at a mechanical low level.

Programming languages define how the code interacts with the hardware, the libraries, and the code that’s already been applied. They don’t affect how the programmer sees their problem, or thinks about the factors involved. They affect the presentation of those thoughts, but that’s at a mechanical low level.

Function-oriented Objects

Given a Cartesian representation of the point (x,y), find its distance from the origin and angle from the x axis.

I’m going to approach this problem using the principles of functional programming. There’s clearly a function that can take us from the coordinates to the displacement, and one that can take us from the coordinates to the angle. Ignoring the implementation for the moment, they look like this:

Point_radius :: float, float -> float
Point_angle  :: float, float -> float

This solution has its problems. I have two interchangeable arguments (both the x and y ordinates are floats) used in independent signatures, how do I make it clear that these are the same thing? How do I ensure that they’re used in the same way?

One tool in the arsenal of the functional programmer is pattern matching. I could create a single entry point with an enumeration that selects the needed operation. Now it’s clear that the operations are related, and there’s a single way to interpret the arguments, guaranteeing consistency.

Point :: float, float, Selector -> float

Good for now, but how extensible is this? What if I need to add an operation that returns a different type (for example a description that returns a string), or one that needs other arguments (for example the projection on to a different vector)? To provide that generality, I’ll use a different tool from the functional toolbox: the higher-order function. Rewrite Point so that instead of returning a float, it returns a function that captures the coordinates and takes any required additional arguments to return a value of the correct type. To avoid cluttering this example with irrelevant details, I’ll give that function a shorthand named type: Method.

Point :: float, float, Selector -> Method

You may want to perform multiple operations on values that represent the same point. Using a final functional programming weapon, partial application, we can capture the coordinates and let you request different operations on the same encapsulated data.

Point :: float, float -> Selector -> Method

Now it’s clear to see that the Point function is a constructor of some type that encapsulates the coordinates representing a given two-dimensional Cartesian point. That type is a function that, upon being given a Selector representing some operation, returns a Method capable of implementing that operation. The function implements message sending, and Points are just objects!

Imagine that we wanted to represent points in a different way, maybe with polar coordinates. We could provide a different function, Point', which captures those:

Point' :: float, float -> Selector -> Method

This function has the same signature as our original function, it too encapsulates the constructor’s arguments (call them instance variables) and returns methods in response to selectors. In other words, Point and Point' are polymorphic: if they have methods for the distance and angle selectors, they can be used interchangeably.

Object-oriented Functions

Write a compiler that takes source code in some language and creates an executable. If it encounters malformed source code, it should report an error and not produce an executable.

Thinking about this with my object-oriented head, I might have a Compiler object with some method #compile(source:String) that returns an optional Executable. If it doesn’t work, then use the #getErrors():List<Error> method to find out what went wrong.

That approach will work (as with most software problems there are infinite ways to skin the same cat), but it’s got some weird design features. What will the getErrors() method do if it’s called before the compile() method? If compile() is called multiple times, do earlier errors get kept or discarded? There’s some odd and unclear temporal coupling here.

To clean that up, use the object-oriented design principle “Tell, don’t ask”. Rather than requesting a list of errors from the compiler, have it tell an error-reporting object about the problems as they occur. How will it know what error reporter to use? That can be passed in, in accordance with another OO principle: dependency inversion.

Compiler#compile(source:String, reporter:ErrorConsumer): Optional<Executable>
ErrorConsumer#reportError(error:Error): void

Now it’s clear that the reporter will receive errors related to the invocation of #compile() that it was passed to, and there’s no need for a separate accessor for the errors. This clears up confusion as to what the stored state represents, as there isn’t anyway.

Another object-oriented tool is the Single Responsibility Principle, which invites us to design objects that have exactly one reason to change. A compiler does not have exactly one reason to change: you might need to target different hardware, change the language syntax, adopt a different executable format. Separating these concerns will yield more cohesive objects.

Tokeniser#tokenise(source:String, reporter:ErrorConsumer): Optional<TokenisedSource>
Compiler#compile(source:TokenisedSource, reporter:ErrorConsumer): Optional<AssemblyProgram>
Assembler#assemble(program:AssemblyProgram, reporter:ErrorConsumer): Optional<BinaryObject>
Linker#link(objects:Array<BinaryObject>, reporter:ErrorConsumer): Optional<Executable>
ErrorConsumer#reportError(error:Error): void

Every class in this system is named Verber, and has a single method, #verb. None of them has any (evident) internal state, they each map their arguments onto return values (with the exception of ErrorConsumer, which is an I/O sink). They’re just stateless functions. Another function is needed to plug them together:

Binder<T,U,V>#bind(T->Optional<U>, U->Optional<V>): (T->Optional<V>)

And now we’ve got a compiler constructed out of functions constructed out of objects.

Brain-oriented programming

Those examples were very abstract, not making any use of specific programming languages. Because software design is not coupled to programming languages, and paradigmatic approaches to programming are constrained ways to think about software design. They’re abstract enough to be separable from the nuts and bolts of the implementation language you choose (whether you’ve already chosen it or not).

Those functions in the Point example could be built using the blocks available in Smalltalk, Ruby or other supposedly object-oriented languages, in which case you’d have objects built out of functions that are themselves built out of objects (which are, of course, built out of functions…). The objects and classes in the Compiler example can easily be closures in a supposedly functional programming language. In fact, closures and blocks are not really too dissimilar.

What conclusions can be derived from all of this? Clearly different programming paradigms are far from exclusive, so the first lesson is that you don’t have to let your choice of programming language dictate your choice of problem solving approach (nor do you really have to do it the other way around). Additionally where the approaches try to solve the same problem, the specific techniques they comprise are complementary or even identical. Both functional and object-oriented programming are about organisation, decomposition and comprehensibility, and using either or even both can help to further those aims.

Ultimately your choice of tools isn’t going to affect your ability to think by much. Whether they help you express your thoughts is a different matter, but while expression is an important part of our work it’s only a small part.

Further Reading

The ideas here were primarily motivated by Uday Reddy’s Objects as Closures: Abstract Semantics of Object-Oriented Languages (weird embedded PDF reader link). In the real-life version of this presentation I also talked a bit about Theorems for Free (actual PDF link) by Philip Wadler, which isn’t so related but is nonetheless very interesting :).

Week three

This is both an international and a hyper-local update. I say international, although I’m still in an airport less than twenty miles from my house. I’m on my way to Florence, to talk about *-oriented programming at Pragma Conference 2015. It’ll be fun to catch up with friends in a new city, and close the conference with what I hope is an informative talk. I’ll put my notes up some time after the talk.

The theme of this week has been friends. I’ve had lots of things I could be doing, and even a few things I should be doing. For the most part I’ve been letting that take second place to spending time with friends. I’ve ended up playing a lot of tunes and singing a lot of songs as a result, and maybe only written four or five paragraphs of useful text.

Speaking of singing and playing, our band is going to play the main spot at Folk in the Barn, Stockton on the 27th. Anyone who’s around the Somerville Arms, Leamington on Tuesday 13th October or Harbury folk club on Thursday 5th November will also discover a subset of the band doing a couple of numbers. International readers will be amused by the provincial nature of this part of the post; a lot of the fun of this break has been connecting with the local area and things that are going on around me.

Week Two

As week one featured an observation of how post-work life was similar to working life, this week’s post is a catalogue of differences. Not all of these differences are huge.

No watch

I own five watches (three wrist, two pocket) but have only worn one once in the last two weeks. Most of the time, I don’t need to know what the time is and don’t need to mark its passing.

No beard

When you get up at 5:30 every morning for the commuter train, putting time into your appearance takes a back seat to getting out of the door quickly. I now shave every day; not particularly important except that it indicates I have time to do so.

No computer

Or at least no sitting at a computer. I’m writing this on a laptop which I’ve just been using to write up the dissertation, but when I’m not doing that, or editing music scores in MuseScore, I’m not “at” a computer. Tempus Fugit was written on my phone: the best computer is the one you have on you.

No tech news

My morning ritual used to involve a lot of RSS tech news feeds, as well as browsing some aggregation sites like to find articles to read. Now that’s all been pared back, apart from a few people who write well. I usually have two or three unread articles every day now, which are more often than not web comics.