Structure and Interpretation of Computer Programmers

I make it easier and faster for you to write high-quality software.

Wednesday, June 24, 2020

It protects. It also promotes and prevents.

I sometimes get asked to review, or “comment on”, the architecture for an app. Often the app already exists, and the architecture documentation consists of nothing more than the source code and the folder structure. Sometimes the app doesn’t exist, and the architecture is a collection of hopes and dreams expressed on a whiteboard. Very, very rarely, both exist.

To effectively review an architecture and make recommendations for improving it, we need much more information than that. We need to know what we’re aiming for, so that we can tell whether the architecture is going to support or hinder those goals.

We start by asking about the functional requirements of the application. Who is using this, what are they using it for, how do they do that? Does the architecture make it easy for the programmers to implement those things, for the testers to validate those things, for whoever deploys and maintains the software to provide those things?

If you see an “architecture” that promotes the choice of technical implementation pattern over the functionality of the system, it’s getting in the way. I don’t need to know that you have three folders of Models, Views and Controllers, or of Actions, Components, and Containers. I need to know that you let people book childrens’ weather forecasters for wild atmospheric physics parties.

We can say the same about non-functional requirements. When I ask what the architecture is supposed to be for, a frequent response is “we need it to scale”. How? Do you want to scale the development team? By doing more things in parallel, or by doing the same things faster, or by requiring more people to achieve the same results? Hold on, did you want to scale the team up or down?

Or did you want to scale the number of concurrent users? Have you tried… y’know, selling the product to people? Many startups in particular need to learn that a CRM is a better tool for scaling their web app than Kubernetes. But anyway, I digress. If you’ve got a plan for getting to a million users, and it’s a realistic plan, does your architecture allow you to do that? Does it show us how to keep that property as we make changes?

Those important things that you want your system to do. The architecture should protect and promote them. It should make it easy to do the right thing, and difficult to regress. It should prevent going off into the weeds, or doing work that counters those goals.

That means that the system’s architecture isn’t really about the technology, it’s about the goals. If you show me a list of npm packages in response to questions about your architecture, you’re not showing me your architecture. Yes, I could build your system using those technologies. But I could probably build anything else, too.

posted by Graham at 18:14  

Tuesday, November 26, 2019

Everyone rejecting everyone else

It’s common in our cooler-than-Agile, post-Agile community to say that Agile teams who “didn’t get it” eschewed good existing practices in their rush to adopt new ways of thinking. We don’t need UML, we’re Agile! Working software over comprehensive documentation!

This short post is a reminder that it ran both ways, and that people used to the earlier ways of thinking also eschewed integrating their tools into the Agile methodology. We don’t need Agile, we’re Model-Driven! Here’s an excerpt from 2004’s UML 2 Toolkit:

Certain object-oriented methods on the market today, such as The Unified Process, might be considered processes. However, other lightweight methods, such as Extreme Programming (XP), are not robust enough to be considered oricesses in the way we use the term here. Although they provide a valuable set of interrelated techniques, they are often referred to as “small m methodologies” rather than as software-engineering processes.

This is in the context of saying that UML supports software-engineering processes in which a sequence of activities produce “documentation…about the system being built” and “a product that solves the initial problems is introduced and delivered”. So XP is not robust enough at doing those things for UML advocates to advocate UML in the context of XP.

So it’s not just that Agilistas abandoned existing practices, but there’s an extent to which existing practitioners abandoned Agilistas too.

posted by Graham at 10:15  

Tuesday, November 5, 2019

On exploding boilers

Throughout our history, it has always been standardisation of components that has enabled creations of greater complexity.

This quote, from Simon Wardley’s finding a path, reminded me of the software industry’s relationship with interchangeable parts.

Brad Cox, in both Object-Oriented Programming: an Evolutionary Approach and Superdistribution, used physical manufacturing analogies (to integrated circuits, and to rifles) to invoke the concept of a “software industrial revolution” that would allow end users to assemble off-the-shelf parts to solve their problems. His “software ICs” built on ideas expressed at least as early as 1968 by Doug McIlroy. Joe Armstrong talked about a universal function registry, so that if someone writes sin/1 everybody else can use it.

Of course we have a lot of reusable components in software engineering now, and we can thank the Free Software movement at least as much as any paradigm of organising programming instructions. CTAN, CPAN, and later repositories act as the “component catalogues” that Cox discussed. If you want to make a computer do something, you can probably find an npm module or a Ruby gem that does most of the work for you. The vast majority of such components have free licenses, it’s rare to pay for a reusable component.

The extent to which they’re “standard parts”, on the model of interchangeable nuts and bolts or integrated circuits, is debatable. Let’s say that you download a package from the NPM. We know that you use it by calling require (or maybe import)…but what does that give you? An object? A constructor? A regular function? Does it run anything as a result of calling require? Does it work in your node/ionic/electron/etc. context? Is it even a lump of regular javascript, or is it a Real, or to have access to a JVM, or some other niche requirement?

Whatever these “standard parts” and however they’re used, you’re probably still doing a bunch of coding. These parts will do computery stuff, or maybe generic behaviour like authentication, date UIs, left-padding strings and the like. Usually we still have to develop ours apps as “engineered” software projects with significant levels of custom coding, to make those “standard parts” actually solve a useful problem. There are still people working for retail companies maintaining online store applications across the four corners of the globe, despite the fact that globes don’t have corners, these things all work the same way, and the risks associated with getting them wrong are significant.

Perhaps this is because software is a distinct thing, and we can never treat it like industrial product manufacturing.

Perhaps this is because our ambition always runs out ahead of our capability. Whatever we can reproducibly build, we’d like to be building something greater.

Perhaps this is because we’re still in the cottage industry stage, where we don’t yet know whether or how to standardise the parts, and occasionally the boilers explode.

posted by Graham at 17:56  

Thursday, October 31, 2019


Having discussed reasons for change with a colleague on my team, we came up with the sprouts of change. Good software is antifragile in the face of changing:

  • Situation
  • People
  • Requirements
  • Organisation
  • Understanding
  • Technology
  • Society

Like any good acronym, it’s really tenuous.

posted by Graham at 14:28  

Wednesday, October 30, 2019


I was just discussing software architecture and next steps with a team building a tool to help analyse MRI images of brains. Most of the questions we asked explored ways to proceed by focussing on change:

  • what if the budget for that commercial component shows up? How would that change the system?
  • what if you find this data source isn’t good enough? How would you find that out?
  • which of these capabilities does the customer find most important? When will they change their minds?

that sort of thing.

We have all sorts of words for planning for, and mitigating the risk of, changes in low-level software design. In fact a book on building maintainable software talks about nothing else, because maintainable software is antifragile software.

But it happened that I wasn’t reading that book at the time, I was reading about high-level design and software architecture. The guide I was reading talked a lot about capturing the requirements and constraints in your software architecture, and this is all important stuff. If someone’s paying for your thing, you need to ensure it can do the things they’re paying for it to do. After all, they’re probably paying to be able to do the things that your software lets them do; they aren’t paying to have some software. Software isn’t real.

However, most of the reason your development will slow down once you’ve got that first version out of the door is that the world (which might be real) changes in ways that it’s hard to adapt your software to. Most of the reason you’re not adding new features is that you’re fixing bugs, i.e. changing the behaviour of the software from one that matches the flawed conception you had of what it should do to one that matches the flawed conception you now have of what it should do.

A good architecture should identify, localise, and separate sources of change in the software system. And then it should probably do whatever you think the customers think they want.

posted by Graham at 14:15  

Thursday, October 24, 2019

Zen and the Art of Software Maintenance

In one part of the book Zen and the Art of Motorcycle Maintenance, which is neither about Zen nor motorcycle maintenance, there are two motorcycles and two riders. John Sutherland is a romanticist who appreciates the external qualities of his motorcycle: its aesthetics, and its use as a vehicle. The narrator is a classicist who appreciates the internal qualities of his motorcycle: its workings, parts, and mechanisms. When Sutherland has a problem with his bike he takes it to a mechanic. When the narrator does, he rationalises about the problem and attempts to discover a solution.

The book, which as its subtitle gives away is “an inquiry into values”, then follows the narrator’s exploration of a third way of considering quality that marries the romantic and classical notions holistically.

Now we come onto software. Software doesn’t exist. At some level, its abstractions and mathematics get translated into a sequence of states of an electronic machine that turns logic into procedure: but even that is a description that’s a few degrees abstracted from what software and computers really do.

Nonetheless, software has external and internal qualities. It has aesthetics and utility, and can be assessed romantically. A decidedly pedestrian word to describe the romanticist view of software is “requirements”, but it’s a common word in software engineering that means the right thing.

Software also has workings, parts, and mechanics. Words from software engineering to describe the classical view of software include architecture, design, clean code, SOLID…

…there are many more of these words! Unsurprisingly, the people who build software and who change software tend to take a classical view of the software, and have a lot more words to describe its internal qualities than its external qualities.

Typically, the people who are paying for software are interested in the romantic view. They want it to work to achieve some goal, and want someone else (us!) to care about what makes it work. Perhaps that’s why so many software teams phrase their requirements as “As a romantic, I want to task so that I can goal.”

Which is to say that making software professionally involves subordinating classical interpretations of quality to romantic interpretations. Which is not to say that a purely-classical viewpoint is unvaluable. It’s just a different thing from teaching a computer somersaults for a paying audience.

And maybe that subordination of our classical view to the customer/gold owner’s romantic view is the source of the principles:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.


Working software is the primary measure of progress.

In fact, this second one is not quite true. It suggests that you could somehow “count software”, and the more (working) software you’ve delivered, the better you’re doing. In fact, romanticism shows us that people only want software in that it enables some process or business opportunity, or makes it more efficient, or reduces errors, or lets them enjoy some downtime, or helps them achieve some other goal. So really progress toward that goal is the primary measure of progress, and working software is a leading metric that we hope tells us how we’re working toward that goal.

So all of those code quality and software architecture things are in support of the external view of the software, which is itself in support of some other, probably non-software-related, goal. And that’s why the cleanliness, or architectural niceness, or whatever classical quality, of the code is not absolute, but depends on how those qualities support the romantic qualities of the code.

Real life comes at you fast, though. When you’re working on version 1, you want to do as little work, as quickly as possible, to get to the point where you can validate that there are enough customers who derive enough value to make the product worthwhile. But by the time you come to work on version 1.0.1, you wish you’d taken the time to make version 1 maintainable and easy to change. Most subsequent versions are a little from column A and a little from column B, as you try new things and iterate on the things that worked.

As fast as possible, but no faster, I guess.

posted by Graham at 23:51  

Wednesday, October 9, 2019

Great Documentation, Great Software

A paraphrased conversation, the other day, between me and a customer of one of my customers:

Me: Are you experienced at working with my customer’s developer APIs?

Them: I always feel like a newbie, because there’s so much stuff. But I always end up finding the docs I’m looking for.

Me: I’m writing the docs.

Them: Well, thanks! :D

Whether you’re writing developer APIs or graphical user interfaces, quality documentation that’s easy to find and use when needed is the best way to turn customers from novices who find the complexity offputting, to novices who know they’ll be able to tackle whatever’s coming their way.

Quality documentation is also useful for improving the quality of the software itself.

Docs-driven development

If you already know about test-driven development, you know that a benefit of TDD as a design tool is that it encourages you to think about your code from the perspective of how it will be used. Rather than implementing an algorithm then exposing an API that you hope will be useful, you design the API that helps solve the problem then implement an algorithm to support the use of that API.

Documentation is another tool for encouraging empathy in design. For every point you have to explain, you get to ask:

  • “why do I have to explain this?”

  • “Is there another way to design this such that I don’t need to tell people about this detail?”

  • “Is the thing that I’m telling people how to do, the thing that they would expect to want to do?”

Dev-driven documentation

The questions listed above can be most effectively answered if documentation is part of your iterative cycle of continuous improvement. Documentation can inform design and development, by pointing out cumbersome or difficult parts of the implementation. Development can inform documentation, by showing where the complexity lies and how to deal with it.

Documentation interacts with other activities, too. Test plans should ensure that they cover examples or walkthroughs from the documentation, so that you know the examples you’re giving to your customers actually work. Documenters should collaborate with testers to ensure a shared understanding of what the software is aiming to achieve.

Documents like API specifications, user manuals, or walkthrough videos should be versioned and built alongside the corresponding versions of the software.

Working software over comprehensive documentation

Throughout these activities, the point is not to generate documentation for its own sake. One office I worked in had a shelf containing several feet of documentation for a UNIX system that I never opened: the online documentation and a couple of cookbook-style books were sufficient.

The reason for putting effort into your software’s documentation is that this effort yields improvements in the software. A more empathetic design, a better-tested implementation, and more confident customers are all steps on the path to higher-quality software, easier and faster. And of course, the Labrary can help you with that.

posted by Graham at 11:35  

Wednesday, August 21, 2019

Maybe you are going to need it

In the beginning, there was the green field. The lead developer, who may have been the only developer, agreed with the product owner (or “the other member of the company” as they were known) what they would build for the first two weeks. Then File->New Project… happened, and they smashed it out of the park.

The amorphous and capricious “market” liked what they had to offer, at least enough to win some seed funding. The team grew, and kept the same cadence: see what we need to do for the next ten business days, do it, celebrate that we did it.

As the company, its customers, and its market mature, things start to slow down. It’s imperceptible at first, because velocity stays constant. The CTO can’t help but think that they get a lot less out of a 13-point story than they used to, but that isn’t a discussion they’re allowed to have. If you convert points into time then you’re doing old waterfall thinking, and we’re an agile team.

Initially the dysfunction manifests in other ways. Developers complain that they don’t get time to refactor, because “the business” doesn’t understand the benefits of clean code. Eventually time is carved out to clean things up, whether in “hardening sprints” or in effort allocated to “engineering stories”. We are getting as much done, as long as you ignore that less of it is being done for the customers.

Stories become task-sliced. Yes, it’s just adding a button, but we need to estimate the adding a component task, the binding the action task, the extending the reducer task, the analytics and management intelligence task. Yes we are getting as much done, as long as you ignore that less of it has observable outcomes.

Rework increases too, as the easy way to fit a feature into the code isn’t the way that customers want to use it. Once again, “the business” is at fault for not being clear about what they need. Customers who were previously flagship wins are now talked about as regressive laggards who don’t share the vision. Stories must have clearer acceptance criteria, the definition of done must be more explicit: but obviously we aren’t talking about a specification document because we’re an agile team. Yes we’re getting as much done, as long as you ignore that a lot of what we got done this fortnight was what we said we’d done last fortnight.

Eventually forward progress becomes near zero. It becomes hard to add new features, indeed hard even to keep up with the competitors. It’s only two years ago that we were five years ahead of them. People start demoing new ideas in separate apps, because there’s no point dreaming about adding them to our flagship project. File->New Project… and start all over again.

What happened to this team? Or really, to these teams, as I’ve seen this story repeated over and over. They misread “responding to change over following a plan” as “we don’t need no stinking plan”.

Even if you don’t know exactly where you are going at any time, you have a good idea where you think you’re going. It might be spread around the company, which is why we need the experts around the table. Some examples of where to find this information:

  • The product owner has a backlog of requested features that have yet to be built.
  • The sales team have a CRM indicating which prospects are hottest, and what they need to offer to close those deals.
  • The marketing director has a roadmap slide they’re presenting at a conference next month.
  • The CTO has budget projections for the next financial year, including headcount changes and how they plan to reorganise the team to incorporate these changes.
  • The CEO knows where they want to position the company in the market over the next two years, and knows which competitors, regulatory changes, and customer behaviours threaten that position and what about them makes them a threat.
  • Countless spreadsheets, databases, and “business intelligence” dashboards across multiple people and departments.

No, we don’t know the future, but we do know which futures are likely and of those, which are desirable. Part of embracing change is to make those futures easier to cope with. The failure mode of many teams is to ignore all futures because we aren’t in any of them yet.
We should be ready for the future we expect, and both humble and adaptable enough to get ready for a different future when things change. Our software should represent our current knowledge of our problem and its solution, including knowledge about likely developments (hey, maybe there’s a reason they call us developers!). Don’t add the things you aren’t going to need, but don’t exclude the possibility of adding them out of spite for a future that may well come to pass.

posted by Graham at 14:10  

Wednesday, June 26, 2019

Longer, fuller stacks

Thinks to self: OK, this “full-stack” project is going to be fairly complex. I need:

  • a database. I don’t need it yet, I’ll defer that.
  • a thing that runs on the server, listens for HTTP requests from a browser, builds responses, and sends them to the browser.
  • a thing that runs on the browser, built out of bits assembled by the server thing, that the user can interact with.

What I actually got was:

  • a thing that runs on the server.
  • a thing that defines the environment for the server.
  • a thing that defines the environment on development machines so that you can run server-development tasks.
  • a thing that turns code that can’t run in a browser into code that can run in a browser.
  • a thing that turns code that can run in a browser into code that does run in real browsers.
  • a headless browser that lets me test browser code.
    • BTW, it doesn’t work with that server environment.
    • a thing that shows how Linux binaries are loaded, to work out how to fix the environment.
    • also BTW, it doesn’t run headless without setting some environment variable
    • a thing that is used for cross-platform native desktop apps, that I can configure to work headless.
  • a thing that builds the bits assembled by the server thing so that the test thing can see the code being tested.

And somehow, people argue that this is about reducing development costs.

posted by Graham at 11:19  

Wednesday, March 6, 2019

Mach and Matchmaker: kernel and language support for object-oriented distributed systems

About this paper

Mach and Matchmaker: kernel and language support for object-oriented distributed systems
, Michael B. Jones and Richard F. Rashid, from the proceedings of OOPSLA ’86.


Yes, 1986 was a long time ago, but the topics of Mach and Matchmaker are still relevant, and I find it interesting to read about its genesis and development. I also find that it helps me put today’s uses – or abandonments – in context.


Two main families of operating systems under development today are still based on the CMU Mach project. Let’s get discussing the HURD out of the way, first. The HURD is based on GNU Mach, which is itself based on the University of Utah’s Mach 4.0 project. GNU Mach is a microkernel, so almost all of the operating system facilities are provided by user-space processes. An interesting implication is that a regular user can create a sub-HURD, an environment with a whole UNIX-like system running within their user account on the host HURD.

Not many people do that, though. HURD is very interesting to read and use, but didn’t fulfil its goal of becoming a free host for the GNU system that made it easy to support hardware. Linux came along, as a free host for the GNU system that made it worthwhile to support hardware. I enjoy using the HURD, but we’ll leave it here.

…because we need to talk about the other operating system family that uses Mach, macOS/iOS/watchOS/tvOS/whatever the thing that runs the touchbar on a Macbook Pro is called OS. These are based on CMU Mach 2.5, for the most part, which is a monolithic kernel. Broadly speaking, Mach was developed by adding bits to the 4.2 (then 4.3) BSD kernel, until it became possible to remove all of the BSD bits and still have a working BSD-like system. Mach 2.5 represents the end of the “add Mach bits to a BSD kernel” part of the process.

Based on an earlier networked environment called Accent, Mach has an object-oriented facility in which object references are called “ports”, and you send them messages by…um, sending them messages. But sending them messages is really hard, because you have to get all the bits of all the parameters in the right place. What you need is…


Originally built for Accent, Matchmaker is an Interface Definition Language in which you describe the messages you want a client and server to use, and it generates procedures for sending the messages and receiving the responses in the client, and receiving the messages and sending the responses in the server.

Being built atop Mach, Matchmaker turns those messages into Mach messages sent between Mach ports. What Mach does to get the messages around is transparent, so it might take a message on one computer and deliver it to a server on a different computer, maybe even running a different architecture.

That transparency was a goal of a lot of object-oriented remote procedure call systems in the 1990s, and by and large fell flat. The reason is Peter Deutsch’s Eight Fallacies of Distributed Computing. Basically you usually want to know when your message is going out over a network, because that changes everything from how likely it is to be received, how likely you are to get a response, to how expensive it will be to send the message.

Matchmaker supported C, Common LISP, Ada, and PERQ Pascal; Accent and Mach messages, and a bunch of different computer architectures. Unfortunately it supported them all through specific knowledge of each, and the paper described here acknowledges how difficult that makes it to work on and proposes future work to clean it up. It’s not clear that future work was ever done; modern Machs all use MIG, an “interim subset” of Matchmaker that only supports C.

Object-oriented design

In my book OOP the Easy Way, I explore the idea that objects are supposed to be small, independent computer programs that communicate over the loosely-coupled channel that is message-sending. Mach and Matchmaker together implement this design. Your objects can be in different languages, on different computers, even in different host operating systems (there were Mach IPC implementations for Mach, obviously, but also VAX Unix and non-Mach BSD). As long as they understand the same format for messages, they can speak to each other.

Consider a Cocoa application. It may be written in Swift or Objective-C or Objective-C++ or Python or whatever. It has a reference to a window, where it draws its views. The app sees that window as an Objective-C object, but the app doesn’t have a connection to the framebuffer to draw windows.

The Window Server has that connection. So when you create a window in your Cocoa application, you actually send a message to the window server and get back a port that represents your window. Messages sent to the window are forwarded to the window server, and events that happen in the UI (like the window being closed or resized) are sent as messages to the application.

Because of the way that Mach can transparently forward messages, it’s theoretically possible for an application on one computer to display its UI on another computer’s window server. In fact, that’s more than a theoretical possibility. NeXTSTEP supported exactly that capability, and an application with the NXHost default set could draw to a window server on a different computer, even one with a different CPU architecture.

This idea of loosely-coupled objects keeps coming up, but particular implementations rarely stay around for long. Mach messages still exist on HURD and Apple’s stuff (both using MIG, rather than Matchmaker), but HURD is tiny and Apple recommend against using Mach or MIG directly, favouring other interfaces like XPC or the traditional UNIX IPC systems that are implemented atop Mach. Similarly, PDO has come and gone, as have CORBA and its descendents DSOM and DOE.

Even within the world of “let’s use HTTP for everything”, SOAP gave way to REST, which gave way to the limited thing you get if you do the CRUD bits of REST without doing the DAP bits. What you learn by understanding Mach and its interfaces is that this scheme can be applied everywhere from an internet service down to an operating system component.

posted by Graham at 13:00  
Next Page »

Powered by WordPress